DreamCraft3D: Hierarchical 3D Generation with Bootstrapped Diffusion
  Prior
We present DreamCraft3D, a hierarchical 3D content generation method that
produces high-fidelity and coherent 3D objects. We tackle the problem by
leveraging a 2D reference image to guide the stages of geometry sculpting and
texture boosting. A central focus of this work is to address the consistency
issue that existing works encounter. To sculpt geometries that render
coherently, we perform score distillation sampling via a view-dependent
diffusion model. This 3D prior, alongside several training strategies,
prioritizes the geometry consistency but compromises the texture fidelity. We
further propose Bootstrapped Score Distillation to specifically boost the
texture. We train a personalized diffusion model, Dreambooth, on the augmented
renderings of the scene, imbuing it with 3D knowledge of the scene being
optimized. The score distillation from this 3D-aware diffusion prior provides
view-consistent guidance for the scene. Notably, through an alternating
optimization of the diffusion prior and 3D scene representation, we achieve
mutually reinforcing improvements: the optimized 3D scene aids in training the
scene-specific diffusion model, which offers increasingly view-consistent
guidance for 3D optimization. The optimization is thus bootstrapped and leads
to substantial texture boosting. With tailored 3D priors throughout the
hierarchical generation, DreamCraft3D generates coherent 3D objects with
photorealistic renderings, advancing the state-of-the-art in 3D content
generation. Code available at https://github.com/deepseek-ai/DreamCraft3D.
6539da7f26df26ecd1350d99
Jingxiang Sun
claimed_verified
2023-10-26T08:16:05.029Z
6456018a95082f722d136946
/avatars/c9f825b9ac0af1e6f615ddcdd9a771ac.svg
jingxiang sun
user
starksun
6539da7f26df26ecd1350d9a
Bo Zhang
claimed_verified
2024-03-12T13:33:57.287Z
6237f3eb4cf571bb1ed4036b
/avatars/cbfb8ddda18e65d18a818db9f176e16e.svg
Bo Zhang
user
zhangmozhe
6539da7f26df26ecd1350d9b
Ruizhi Shao
admin_assigned
2023-10-26T07:47:45.986Z
64785ad575a9a0cfabcf5083
/avatars/5b8526bf2d8389310ca1799655cbbcd8.svg
Ruizhi Shao
user
DSaurus
6539da7f26df26ecd1350d9c
Lizhen Wang
admin_assigned
2023-10-26T07:47:55.693Z
63451d2dfeba4bdba59cf9b1
/avatars/ff5dd2f3b502c54e7c3e2512c3e98b28.svg
Lizhen Wang
user
wanglz14
6539da7f26df26ecd1350d9d
Wen Liu
admin_assigned
2023-10-26T07:50:50.243Z
63198e8802fb322037332f2d
/avatars/d3f9c206e387df35beb0ed0ef1cdf865.svg
Wen Liu
user
doubility123
6539da7f26df26ecd1350d9e
Zhenda Xie
admin_assigned
2023-10-26T07:48:38.640Z
6539f6ea26df26ecd1393c37
https://cdn-avatars.huggingface.co/v1/production/uploads/6539f6ea26df26ecd1393c37/9VJusLLAiLhUxfBAFggpF.jpeg
Zhenda Xie
user
zdaxie
6539da7f26df26ecd1350d9f
Yebin Liu
admin_assigned
2023-10-26T07:48:55.762Z
62e14dbe4db2175cd2735a80
/avatars/e6385dcedcb97e1b36281b49210321aa.svg
Yebin Liu
user
YebinLiu
DreamCraft3D: Hierarchical 3D Generation with Bootstrapped Diffusion
  Prior
We present DreamCraft3D, a hierarchical 3D content generation method that
produces high-fidelity and coherent 3D objects. We tackle the problem by
leveraging a 2D reference image to guide the stages of geometry sculpting and
texture boosting. A central focus of this work is to address the consistency
issue that existing works encounter. To sculpt geometries that render
coherently, we perform score distillation sampling via a view-dependent
diffusion model. This 3D prior, alongside several training strategies,
prioritizes the geometry consistency but compromises the texture fidelity. We
further propose Bootstrapped Score Distillation to specifically boost the
texture. We train a personalized diffusion model, Dreambooth, on the augmented
renderings of the scene, imbuing it with 3D knowledge of the scene being
optimized. The score distillation from this 3D-aware diffusion prior provides
view-consistent guidance for the scene. Notably, through an alternating
optimization of the diffusion prior and 3D scene representation, we achieve
mutually reinforcing improvements: the optimized 3D scene aids in training the
scene-specific diffusion model, which offers increasingly view-consistent
guidance for 3D optimization. The optimization is thus bootstrapped and leads
to substantial texture boosting. With tailored 3D priors throughout the
hierarchical generation, DreamCraft3D generates coherent 3D objects with
photorealistic renderings, advancing the state-of-the-art in 3D content
generation. Code available at https://github.com/deepseek-ai/DreamCraft3D.
DreamCraft3D uses a hierarchical approach with diffusion models and tailored 3D priors to generate high-fidelity and coherent 3D objects from 2D references, addressing geometry consistency and texture fidelity.
diffusion model score distillation sampling Bootstrapped Score Distillation Dreambooth 3D prior alternating optimization view-dependent diffusion model 3D scene representation 3D content generation