Difix3D+: Improving 3D Reconstructions with Single-Step Diffusion Models
Neural Radiance Fields and 3D Gaussian Splatting have revolutionized 3D
reconstruction and novel-view synthesis task. However, achieving photorealistic
rendering from extreme novel viewpoints remains challenging, as artifacts
persist across representations. In this work, we introduce Difix3D+, a novel
pipeline designed to enhance 3D reconstruction and novel-view synthesis through
single-step diffusion models. At the core of our approach is Difix, a
single-step image diffusion model trained to enhance and remove artifacts in
rendered novel views caused by underconstrained regions of the 3D
representation. Difix serves two critical roles in our pipeline. First, it is
used during the reconstruction phase to clean up pseudo-training views that are
rendered from the reconstruction and then distilled back into 3D. This greatly
enhances underconstrained regions and improves the overall 3D representation
quality. More importantly, Difix also acts as a neural enhancer during
inference, effectively removing residual artifacts arising from imperfect 3D
supervision and the limited capacity of current reconstruction models. Difix3D+
is a general solution, a single model compatible with both NeRF and 3DGS
representations, and it achieves an average 2times improvement in FID score
over baselines while maintaining 3D consistency.
67c694febdab31ec59fea175
Jay Zhangjie Wu
claimed_verified
2025-03-04T08:34:53.874Z
633aaf695df91da9cea92960
https://cdn-avatars.huggingface.co/v1/production/uploads/633aaf695df91da9cea92960/h1oOWUudbdlsd63Q5w0hM.png
Jay Wu
user
jayw
67c694febdab31ec59fea176
Yuxuan Zhang
67c694febdab31ec59fea177
Haithem Turki
admin_assigned
2025-03-04T09:13:26.878Z
656e000253703dd78fd072a9
/avatars/6702ba8fabe3d08884aa757f90cea333.svg
Haithem Turki
user
hturki
67c694febdab31ec59fea178
Xuanchi Ren
admin_assigned
2025-03-04T09:13:33.467Z
658529d61c461dfe88afe8e8
/avatars/a22c1b07d28c2662833c462c6537d835.svg
Xuanchi Ren
user
xrenaa
67c694febdab31ec59fea179
Jun Gao
67c694febdab31ec59fea17a
Mike Zheng Shou
admin_assigned
2025-03-04T09:27:21.825Z
661ab3da2b14565c7acccf5c
/avatars/fa4fc03664803e02aede4d4c3d50b393.svg
Mike Zheng Shou
user
AnalMom
67c694febdab31ec59fea17b
Sanja Fidler
67c694febdab31ec59fea17c
Zan Gojcic
admin_assigned
2025-03-04T09:27:34.034Z
6366cda3361a96184dc22139
/avatars/d8a88c84cb5f69e69dd038674a29be89.svg
Zan Gojcic
user
zgojcic
67c694febdab31ec59fea17d
Huan Ling
Difix3D+: Improving 3D Reconstructions with Single-Step Diffusion Models
Neural Radiance Fields and 3D Gaussian Splatting have revolutionized 3D
reconstruction and novel-view synthesis task. However, achieving photorealistic
rendering from extreme novel viewpoints remains challenging, as artifacts
persist across representations. In this work, we introduce Difix3D+, a novel
pipeline designed to enhance 3D reconstruction and novel-view synthesis through
single-step diffusion models. At the core of our approach is Difix, a
single-step image diffusion model trained to enhance and remove artifacts in
rendered novel views caused by underconstrained regions of the 3D
representation. Difix serves two critical roles in our pipeline. First, it is
used during the reconstruction phase to clean up pseudo-training views that are
rendered from the reconstruction and then distilled back into 3D. This greatly
enhances underconstrained regions and improves the overall 3D representation
quality. More importantly, Difix also acts as a neural enhancer during
inference, effectively removing residual artifacts arising from imperfect 3D
supervision and the limited capacity of current reconstruction models. Difix3D+
is a general solution, a single model compatible with both NeRF and 3DGS
representations, and it achieves an average 2times improvement in FID score
over baselines while maintaining 3D consistency.
Difix3D+, a single-step diffusion model pipeline, enhances 3D reconstruction and novel-view synthesis by improving photorealism and reducing artifacts from extreme viewpoints.
Neural Radiance Fields 3D Gaussian Splatting diffusion models 3D reconstruction novel-view synthesis photorealistic rendering underconstrained regions pseudo-training views neural enhancer FID score 3D consistency