ReconFusion: 3D Reconstruction with Diffusion Priors
3D reconstruction methods such as Neural Radiance Fields (NeRFs) excel at
rendering photorealistic novel views of complex scenes. However, recovering a
high-quality NeRF typically requires tens to hundreds of input images,
resulting in a time-consuming capture process. We present ReconFusion to
reconstruct real-world scenes using only a few photos. Our approach leverages a
diffusion prior for novel view synthesis, trained on synthetic and multiview
datasets, which regularizes a NeRF-based 3D reconstruction pipeline at novel
camera poses beyond those captured by the set of input images. Our method
synthesizes realistic geometry and texture in underconstrained regions while
preserving the appearance of observed regions. We perform an extensive
evaluation across various real-world datasets, including forward-facing and
360-degree scenes, demonstrating significant performance improvements over
previous few-view NeRF reconstruction approaches.
656fe23ae542334e137088c0
Rundi Wu
admin_assigned
2023-12-06T12:47:00.925Z
63823d59a18d1ca09653aa21
/avatars/151e1ca46627abdd99c27945ea2fa441.svg
Rundi Wu
user
rundi
656fe23ae542334e137088c1
Ben Mildenhall
656fe23ae542334e137088c2
Philipp Henzler
admin_assigned
2023-12-06T12:47:20.237Z
6386cc1fb3ef0523c6410261
/avatars/f3e950e9044345130becd4b888d488a0.svg
P Henzler
user
solidst8ip
656fe23ae542334e137088c3
Keunhong Park
656fe23ae542334e137088c4
Ruiqi Gao
admin_assigned
2023-12-06T12:47:40.628Z
6137d46d1c0a1ca53676925f
/avatars/f3d884a3971a858adc49e0f3b1932078.svg
Gao
user
Ruiqi
656fe23ae542334e137088c5
Daniel Watson
admin_assigned
2023-12-06T12:47:47.781Z
644f2bb667c9458c913248f8
/avatars/4065b41f7f3568def4a63c44c10670c4.svg
Daniel Watson
user
neftor
656fe23ae542334e137088c6
Pratul P. Srinivasan
656fe23ae542334e137088c7
Dor Verbin
admin_assigned
2023-12-06T12:48:01.130Z
6315b0fc915d0b806830b424
/avatars/e25a7c8df2364c6e8024ec5c646a2889.svg
Dor Verbin
user
dorverbin
656fe23ae542334e137088c8
Jonathan T. Barron
656fe23ae542334e137088c9
Ben Poole
admin_assigned
2023-12-06T12:48:20.557Z
63213768d2d45f315183abc3
/avatars/521f52f430b73e514afb06e3a9d6779d.svg
Ben Poole
user
doinkda
656fe23ae542334e137088ca
Aleksander Holynski
admin_assigned
2023-12-06T12:48:26.923Z
62f6e2792e53c2efd33faa92
/avatars/85d94b4022577747b8d2d10a82c2f3c7.svg
Aleksander Holynski
user
holynski
ReconFusion: 3D Reconstruction with Diffusion Priors
3D reconstruction methods such as Neural Radiance Fields (NeRFs) excel at
rendering photorealistic novel views of complex scenes. However, recovering a
high-quality NeRF typically requires tens to hundreds of input images,
resulting in a time-consuming capture process. We present ReconFusion to
reconstruct real-world scenes using only a few photos. Our approach leverages a
diffusion prior for novel view synthesis, trained on synthetic and multiview
datasets, which regularizes a NeRF-based 3D reconstruction pipeline at novel
camera poses beyond those captured by the set of input images. Our method
synthesizes realistic geometry and texture in underconstrained regions while
preserving the appearance of observed regions. We perform an extensive
evaluation across various real-world datasets, including forward-facing and
360-degree scenes, demonstrating significant performance improvements over
previous few-view NeRF reconstruction approaches.
ReconFusion uses a diffusion prior to synthesize realistic geometry and texture with sparse input images for 3D reconstruction, outperforming previous few-view NeRF methods.
Neural Radiance Fields diffusion prior novel view synthesis 3D reconstruction underconstrained regions observed regions forward-facing 360-degree scenes