Stable Score Distillation for High-Quality 3D Generation
Score Distillation Sampling (SDS) has exhibited remarkable performance in
conditional 3D content generation. However, a comprehensive understanding of
the SDS formulation is still lacking, hindering the development of 3D
generation. In this work, we present an interpretation of SDS as a combination
of three functional components: mode-disengaging, mode-seeking and
variance-reducing terms, and analyze the properties of each. We show that
problems such as over-smoothness and color-saturation result from the intrinsic
deficiency of the supervision terms and reveal that the variance-reducing term
introduced by SDS is sub-optimal. Additionally, we shed light on the adoption
of large Classifier-Free Guidance (CFG) scale for 3D generation. Based on the
analysis, we propose a simple yet effective approach named Stable Score
Distillation (SSD) which strategically orchestrates each term for high-quality
3D generation. Extensive experiments validate the efficacy of our approach,
demonstrating its ability to generate high-fidelity 3D content without
succumbing to issues such as over-smoothness and over-saturation, even under
low CFG conditions with the most challenging NeRF representation.
657fd0d1eaad53ff67ef9f9f
Boshi Tang
657fd0d1eaad53ff67ef9fa0
Jianan Wang
657fd0d1eaad53ff67ef9fa1
Zhiyong Wu
admin_assigned
2023-12-18T11:28:22.095Z
6487e877b541258e038ea921
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/jXU7VV9V7_X530IOOCvVZ.jpeg
wuzhiyong
user
zhiyongdezhi
657fd0d1eaad53ff67ef9fa2
Lei Zhang
Stable Score Distillation for High-Quality 3D Generation
Score Distillation Sampling (SDS) has exhibited remarkable performance in
conditional 3D content generation. However, a comprehensive understanding of
the SDS formulation is still lacking, hindering the development of 3D
generation. In this work, we present an interpretation of SDS as a combination
of three functional components: mode-disengaging, mode-seeking and
variance-reducing terms, and analyze the properties of each. We show that
problems such as over-smoothness and color-saturation result from the intrinsic
deficiency of the supervision terms and reveal that the variance-reducing term
introduced by SDS is sub-optimal. Additionally, we shed light on the adoption
of large Classifier-Free Guidance (CFG) scale for 3D generation. Based on the
analysis, we propose a simple yet effective approach named Stable Score
Distillation (SSD) which strategically orchestrates each term for high-quality
3D generation. Extensive experiments validate the efficacy of our approach,
demonstrating its ability to generate high-fidelity 3D content without
succumbing to issues such as over-smoothness and over-saturation, even under
low CFG conditions with the most challenging NeRF representation.
The paper interprets Score Distillation Sampling as a combination of mode-disengaging, mode-seeking, and variance-reducing terms to improve 3D content generation by proposing a Stable Score Distillation approach, addressing issues like over-smoothness and color-saturation.
Score Distillation Sampling mode-disengaging mode-seeking variance-reducing Classifier-Free Guidance Stable Score Distillation NeRF representation