ComboVerse: Compositional 3D Assets Creation Using Spatially-Aware
  Diffusion Guidance
Generating high-quality 3D assets from a given image is highly desirable in
various applications such as AR/VR. Recent advances in single-image 3D
generation explore feed-forward models that learn to infer the 3D model of an
object without optimization. Though promising results have been achieved in
single object generation, these methods often struggle to model complex 3D
assets that inherently contain multiple objects. In this work, we present
ComboVerse, a 3D generation framework that produces high-quality 3D assets with
complex compositions by learning to combine multiple models. 1) We first
perform an in-depth analysis of this ``multi-object gap'' from both model and
data perspectives. 2) Next, with reconstructed 3D models of different objects,
we seek to adjust their sizes, rotation angles, and locations to create a 3D
asset that matches the given image. 3) To automate this process, we apply
spatially-aware score distillation sampling (SSDS) from pretrained diffusion
models to guide the positioning of objects. Our proposed framework emphasizes
spatial alignment of objects, compared with standard score distillation
sampling, and thus achieves more accurate results. Extensive experiments
validate ComboVerse achieves clear improvements over existing methods in
generating compositional 3D assets.
65fa45725355a52c782a6e9d
Yongwei Chen
claimed_verified
2024-11-26T21:21:43.406Z
6458c809b1202755ec5cf33a
/avatars/0cd92dbb4e85484926c65e5582d89e55.svg
cyw-3d
user
cyw-3d
65fa45725355a52c782a6e9e
Tengfei Wang
admin_assigned
2024-03-21T13:04:09.578Z
62e7c26236a8e8a827ff0891
/avatars/15648407233bf5757262dbc04464d02e.svg
Tengfei Wang
user
tfwang
65fa45725355a52c782a6e9f
Tong Wu
claimed_verified
2024-12-11T09:24:40.770Z
6310929fda0aaad2f70397a1
https://cdn-avatars.huggingface.co/v1/production/uploads/6310929fda0aaad2f70397a1/YoHeYZezmCOXwL-qsiZOc.png
Tong Wu
user
wutong16
65fa45725355a52c782a6ea0
Xingang Pan
admin_assigned
2024-03-21T13:05:04.811Z
646758d6a9b4610868a138ff
https://cdn-avatars.huggingface.co/v1/production/uploads/646758d6a9b4610868a138ff/fNBlK5g4g7vlPAR1bQimZ.png
Xingang Pan
user
Xingang-Pan
65fa45725355a52c782a6ea1
Kui Jia
65fa45725355a52c782a6ea2
Ziwei Liu
admin_assigned
2024-03-21T13:05:19.972Z
62ab1ac1d48b4d8b048a3473
https://cdn-avatars.huggingface.co/v1/production/uploads/1656826685333-62ab1ac1d48b4d8b048a3473.png
Ziwei Liu
user
liuziwei7
ComboVerse: Compositional 3D Assets Creation Using Spatially-Aware
  Diffusion Guidance
Generating high-quality 3D assets from a given image is highly desirable in
various applications such as AR/VR. Recent advances in single-image 3D
generation explore feed-forward models that learn to infer the 3D model of an
object without optimization. Though promising results have been achieved in
single object generation, these methods often struggle to model complex 3D
assets that inherently contain multiple objects. In this work, we present
ComboVerse, a 3D generation framework that produces high-quality 3D assets with
complex compositions by learning to combine multiple models. 1) We first
perform an in-depth analysis of this ``multi-object gap'' from both model and
data perspectives. 2) Next, with reconstructed 3D models of different objects,
we seek to adjust their sizes, rotation angles, and locations to create a 3D
asset that matches the given image. 3) To automate this process, we apply
spatially-aware score distillation sampling (SSDS) from pretrained diffusion
models to guide the positioning of objects. Our proposed framework emphasizes
spatial alignment of objects, compared with standard score distillation
sampling, and thus achieves more accurate results. Extensive experiments
validate ComboVerse achieves clear improvements over existing methods in
generating compositional 3D assets.
ComboVerse generates high-quality 3D assets with complex compositions by combining multiple reconstructed 3D models using spatially-aware score distillation sampling.
feed-forward models multi-object gap reconstructed 3D models spatial alignment spatially-aware score distillation sampling pretrained diffusion models