HiFA: High-fidelity Text-to-3D with Advanced Diffusion Guidance
Automatic text-to-3D synthesis has achieved remarkable advancements through
the optimization of 3D models. Existing methods commonly rely on pre-trained
text-to-image generative models, such as diffusion models, providing scores for
2D renderings of Neural Radiance Fields (NeRFs) and being utilized for
optimizing NeRFs. However, these methods often encounter artifacts and
inconsistencies across multiple views due to their limited understanding of 3D
geometry. To address these limitations, we propose a reformulation of the
optimization loss using the diffusion prior. Furthermore, we introduce a novel
training approach that unlocks the potential of the diffusion prior. To improve
3D geometry representation, we apply auxiliary depth supervision for
NeRF-rendered images and regularize the density field of NeRFs. Extensive
experiments demonstrate the superiority of our method over prior works,
resulting in advanced photo-realism and improved multi-view consistency.
6476bbe183d4fdaedddf9100
Joseph Zhu
admin_assigned
2023-06-01T08:12:11.389Z
60be3d8f2d0691c26ddabf22
/avatars/9eddff30bb54277a6877cecf517ce8b5.svg
Joseph Zhu
user
JunzheJosephZhu
6476bbe183d4fdaedddf9101
Peiye Zhuang
admin_assigned
2023-06-01T08:13:27.389Z
63e473cf9db5da2dc1f4e9ff
/avatars/6973b4c959ef8bb72c40c611d384a78a.svg
peiye zhuang
user
Kelest
HiFA: High-fidelity Text-to-3D with Advanced Diffusion Guidance
Automatic text-to-3D synthesis has achieved remarkable advancements through
the optimization of 3D models. Existing methods commonly rely on pre-trained
text-to-image generative models, such as diffusion models, providing scores for
2D renderings of Neural Radiance Fields (NeRFs) and being utilized for
optimizing NeRFs. However, these methods often encounter artifacts and
inconsistencies across multiple views due to their limited understanding of 3D
geometry. To address these limitations, we propose a reformulation of the
optimization loss using the diffusion prior. Furthermore, we introduce a novel
training approach that unlocks the potential of the diffusion prior. To improve
3D geometry representation, we apply auxiliary depth supervision for
NeRF-rendered images and regularize the density field of NeRFs. Extensive
experiments demonstrate the superiority of our method over prior works,
resulting in advanced photo-realism and improved multi-view consistency.
Reformulating optimization loss with diffusion prior and introducing depth supervision improve text-to-3D synthesis quality and multi-view consistency.
diffusion models Neural Radiance Fields (NeRFs) diffusion prior depth supervision density field