Text-to-3D Generation with Bidirectional Diffusion using both 2D and 3D
  priors
Most 3D generation research focuses on up-projecting 2D foundation models
into the 3D space, either by minimizing 2D Score Distillation Sampling (SDS)
loss or fine-tuning on multi-view datasets. Without explicit 3D priors, these
methods often lead to geometric anomalies and multi-view inconsistency.
Recently, researchers have attempted to improve the genuineness of 3D objects
by directly training on 3D datasets, albeit at the cost of low-quality texture
generation due to the limited texture diversity in 3D datasets. To harness the
advantages of both approaches, we propose Bidirectional Diffusion(BiDiff), a
unified framework that incorporates both a 3D and a 2D diffusion process, to
preserve both 3D fidelity and 2D texture richness, respectively. Moreover, as a
simple combination may yield inconsistent generation results, we further bridge
them with novel bidirectional guidance. In addition, our method can be used as
an initialization of optimization-based models to further improve the quality
of 3D model and efficiency of optimization, reducing the generation process
from 3.4 hours to 20 minutes. Experimental results have shown that our model
achieves high-quality, diverse, and scalable 3D generation. Project website:
https://bidiff.github.io/.
657681d9cbd807a02e03b656
Lihe Ding
admin_assigned
2023-12-11T08:48:34.674Z
64190a6c0956be7233a5f317
/avatars/8590515089bc31a168bc2da2c7516dfb.svg
Lihe Ding
user
Lihe
657681d9cbd807a02e03b657
Shaocong Dong
admin_assigned
2023-12-11T08:48:40.926Z
63ae91af2314b93f9e6dde42
/avatars/792ce138cbee85b8754fdcec7fb1ff52.svg
Shaocong Dong
user
dscdyc
657681d9cbd807a02e03b658
Zhanpeng Huang
admin_assigned
2023-12-11T08:48:53.713Z
6351129285bdb764f6aa4c8a
/avatars/cfc32d19e038caedd65180eb07f87881.svg
HuangZhanpeng
user
RyanHZP
657681d9cbd807a02e03b659
Zibin Wang
657681d9cbd807a02e03b65a
Yiyuan Zhang
claimed_verified
2024-01-29T08:28:52.326Z
63176933b58b0184630d2c74
https://cdn-avatars.huggingface.co/v1/production/uploads/63176933b58b0184630d2c74/53b5EASwW76zeyyqeJA3O.jpeg
Yiyuan Zhang
user
Yiyuan
657681d9cbd807a02e03b65b
Kaixiong Gong
admin_assigned
2023-12-11T08:49:18.537Z
642e427f6748dd4f8eeb2f38
/avatars/07158ff6aa1803c846403594c5d55a34.svg
Kaixiong Gong
user
kxgong
657681d9cbd807a02e03b65c
Dan Xu
claimed_verified
2025-07-15T19:16:10.543Z
66feab48651e00e22f33222e
/avatars/7344377e2c796c7ec85194bb2fc78521.svg
Dan Xu
user
danxuhk
657681d9cbd807a02e03b65d
Tianfan Xue
Text-to-3D Generation with Bidirectional Diffusion using both 2D and 3D
  priors
Most 3D generation research focuses on up-projecting 2D foundation models
into the 3D space, either by minimizing 2D Score Distillation Sampling (SDS)
loss or fine-tuning on multi-view datasets. Without explicit 3D priors, these
methods often lead to geometric anomalies and multi-view inconsistency.
Recently, researchers have attempted to improve the genuineness of 3D objects
by directly training on 3D datasets, albeit at the cost of low-quality texture
generation due to the limited texture diversity in 3D datasets. To harness the
advantages of both approaches, we propose Bidirectional Diffusion(BiDiff), a
unified framework that incorporates both a 3D and a 2D diffusion process, to
preserve both 3D fidelity and 2D texture richness, respectively. Moreover, as a
simple combination may yield inconsistent generation results, we further bridge
them with novel bidirectional guidance. In addition, our method can be used as
an initialization of optimization-based models to further improve the quality
of 3D model and efficiency of optimization, reducing the generation process
from 3.4 hours to 20 minutes. Experimental results have shown that our model
achieves high-quality, diverse, and scalable 3D generation. Project website:
https://bidiff.github.io/.
BiDiff combines 3D and 2D diffusion processes with bidirectional guidance to generate high-quality, diverse, and scalable 3D models efficiently.
3D generation 2D foundation models Score Distillation Sampling SDS multi-view datasets 3D priors geometric anomalies multi-view inconsistency 3D diffusion 2D diffusion bidirectional guidance optimization-based models