IDArb: Intrinsic Decomposition for Arbitrary Number of Input Views and
  Illuminations
Capturing geometric and material information from images remains a
fundamental challenge in computer vision and graphics. Traditional
optimization-based methods often require hours of computational time to
reconstruct geometry, material properties, and environmental lighting from
dense multi-view inputs, while still struggling with inherent ambiguities
between lighting and material. On the other hand, learning-based approaches
leverage rich material priors from existing 3D object datasets but face
challenges with maintaining multi-view consistency. In this paper, we introduce
IDArb, a diffusion-based model designed to perform intrinsic decomposition on
an arbitrary number of images under varying illuminations. Our method achieves
accurate and multi-view consistent estimation on surface normals and material
properties. This is made possible through a novel cross-view, cross-domain
attention module and an illumination-augmented, view-adaptive training
strategy. Additionally, we introduce ARB-Objaverse, a new dataset that provides
large-scale multi-view intrinsic data and renderings under diverse lighting
conditions, supporting robust training. Extensive experiments demonstrate that
IDArb outperforms state-of-the-art methods both qualitatively and
quantitatively. Moreover, our approach facilitates a range of downstream tasks,
including single-image relighting, photometric stereo, and 3D reconstruction,
highlighting its broad applications in realistic 3D content creation.
6760ee8b8cd4d1c2b6f45f23
Zhibing Li
claimed_verified
2024-12-17T08:03:46.852Z
632a80706813868fa4a649e3
https://cdn-avatars.huggingface.co/v1/production/uploads/632a80706813868fa4a649e3/MbTsAYGadNwS3-G5vxEnm.jpeg
Zhibing LI
user
lizb6626
6760ee8b8cd4d1c2b6f45f24
Tong Wu
admin_assigned
2024-12-17T09:44:25.382Z
62fae9328e137d7c4b896498
/avatars/1bda39dec585c099417cc9daa9f53c42.svg
Tong Wu
user
tongwu2020
6760ee8b8cd4d1c2b6f45f25
Jing Tan
admin_assigned
2024-12-17T09:44:07.182Z
65367c40061949598892dbdc
/avatars/4baf27263841471cbd5f629a8b99424d.svg
Jing Tan
user
jingtan
6760ee8b8cd4d1c2b6f45f26
Mengchen Zhang
admin_assigned
2024-12-17T09:43:09.687Z
64de20c5808492ba6e65d124
https://cdn-avatars.huggingface.co/v1/production/uploads/64de20c5808492ba6e65d124/58IX_TI5vJw73qS1knw56.jpeg
Zhang Mengchen
user
Dubhe-zmc
6760ee8b8cd4d1c2b6f45f27
Jiaqi Wang
admin_assigned
2024-12-17T09:42:54.901Z
64638c4d51fa6e63060521b5
/avatars/c863ace5b1dc788a341bcf4ddbdfaec1.svg
JIaqi
user
Jiaqiwang
6760ee8b8cd4d1c2b6f45f28
Dahua Lin
admin_assigned
2024-12-17T09:42:07.153Z
636317ed80c1a705a6eff396
/avatars/3db090e101b916d9256d0d3e043db71d.svg
Dahua Lin
user
lindahua
IDArb: Intrinsic Decomposition for Arbitrary Number of Input Views and
  Illuminations
Capturing geometric and material information from images remains a
fundamental challenge in computer vision and graphics. Traditional
optimization-based methods often require hours of computational time to
reconstruct geometry, material properties, and environmental lighting from
dense multi-view inputs, while still struggling with inherent ambiguities
between lighting and material. On the other hand, learning-based approaches
leverage rich material priors from existing 3D object datasets but face
challenges with maintaining multi-view consistency. In this paper, we introduce
IDArb, a diffusion-based model designed to perform intrinsic decomposition on
an arbitrary number of images under varying illuminations. Our method achieves
accurate and multi-view consistent estimation on surface normals and material
properties. This is made possible through a novel cross-view, cross-domain
attention module and an illumination-augmented, view-adaptive training
strategy. Additionally, we introduce ARB-Objaverse, a new dataset that provides
large-scale multi-view intrinsic data and renderings under diverse lighting
conditions, supporting robust training. Extensive experiments demonstrate that
IDArb outperforms state-of-the-art methods both qualitatively and
quantitatively. Moreover, our approach facilitates a range of downstream tasks,
including single-image relighting, photometric stereo, and 3D reconstruction,
highlighting its broad applications in realistic 3D content creation.
IDArb, a diffusion-based model, accurately decomposes geometric and material information from multiple images under varying lights, improving consistency and performance compared to traditional methods with the help of ARB-Objaverse dataset.
diffusion-based model intrinsic decomposition cross-view cross-domain attention module illumination-augmented view-adaptive training ARB-Objaverse single-image relighting photometric stereo 3D reconstruction