Wonder3D: Single Image to 3D using Cross-Domain Diffusion
In this work, we introduce Wonder3D, a novel method for efficiently
generating high-fidelity textured meshes from single-view images.Recent methods
based on Score Distillation Sampling (SDS) have shown the potential to recover
3D geometry from 2D diffusion priors, but they typically suffer from
time-consuming per-shape optimization and inconsistent geometry. In contrast,
certain works directly produce 3D information via fast network inferences, but
their results are often of low quality and lack geometric details. To
holistically improve the quality, consistency, and efficiency of image-to-3D
tasks, we propose a cross-domain diffusion model that generates multi-view
normal maps and the corresponding color images. To ensure consistency, we
employ a multi-view cross-domain attention mechanism that facilitates
information exchange across views and modalities. Lastly, we introduce a
geometry-aware normal fusion algorithm that extracts high-quality surfaces from
the multi-view 2D representations. Our extensive evaluations demonstrate that
our method achieves high-quality reconstruction results, robust generalization,
and reasonably good efficiency compared to prior works.
6538b453c8b0cccf1f3fe149
Xiaoxiao Long
6538b453c8b0cccf1f3fe14a
Yuan-Chen Guo
admin_assigned
2023-10-26T08:00:11.088Z
6346aaa3f06b237ba4e297b0
/avatars/5acb986e993eab1461200f3e9d99d022.svg
Yuan-Chen Guo
user
bennyguo
6538b453c8b0cccf1f3fe14b
Cheng Lin
6538b453c8b0cccf1f3fe14c
Yuan Liu
admin_assigned
2023-10-26T08:01:34.741Z
64d47a7a508a6313e33faedd
https://cdn-avatars.huggingface.co/v1/production/uploads/64d47a7a508a6313e33faedd/8DrbXdxA-sKZb4jgNhvY1.jpeg
Yuan Liu
user
YuanLiuuuuuu
6538b453c8b0cccf1f3fe14d
Zhiyang Dou
admin_assigned
2023-10-26T08:01:41.723Z
645223fb01d7bd9555ea399a
https://cdn-avatars.huggingface.co/v1/production/uploads/645223fb01d7bd9555ea399a/fVR7XmGg6pMSRKx9sEdvT.png
Zhiyang Dou
user
frankzydou
6538b453c8b0cccf1f3fe14e
Lingjie Liu
6538b453c8b0cccf1f3fe14f
Yuexin Ma
6538b453c8b0cccf1f3fe150
Song-Hai Zhang
6538b453c8b0cccf1f3fe151
Marc Habermann
claimed_verified
2024-08-29T08:07:05.449Z
66d02ac970b7b22757b2c631
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/-71YM7txHBu67bfS_Re8-.jpeg
Marc Habermann
user
MarcHabermann
6538b453c8b0cccf1f3fe152
Christian Theobalt
6538b453c8b0cccf1f3fe153
Wenping Wang
Wonder3D: Single Image to 3D using Cross-Domain Diffusion
In this work, we introduce Wonder3D, a novel method for efficiently
generating high-fidelity textured meshes from single-view images.Recent methods
based on Score Distillation Sampling (SDS) have shown the potential to recover
3D geometry from 2D diffusion priors, but they typically suffer from
time-consuming per-shape optimization and inconsistent geometry. In contrast,
certain works directly produce 3D information via fast network inferences, but
their results are often of low quality and lack geometric details. To
holistically improve the quality, consistency, and efficiency of image-to-3D
tasks, we propose a cross-domain diffusion model that generates multi-view
normal maps and the corresponding color images. To ensure consistency, we
employ a multi-view cross-domain attention mechanism that facilitates
information exchange across views and modalities. Lastly, we introduce a
geometry-aware normal fusion algorithm that extracts high-quality surfaces from
the multi-view 2D representations. Our extensive evaluations demonstrate that
our method achieves high-quality reconstruction results, robust generalization,
and reasonably good efficiency compared to prior works.
Wonder3D leverages cross-domain diffusion models and geometry-aware normal fusion for high-quality, efficient, and consistent 3D mesh generation from single-view images.
Score Distillation Sampling diffusion models multi-view normal maps cross-domain attention mechanism geometry-aware normal fusion 3D mesh generation