Geometry Image Diffusion: Fast and Data-Efficient Text-to-3D with
  Image-Based Surface Representation
Generating high-quality 3D objects from textual descriptions remains a
challenging problem due to computational cost, the scarcity of 3D data, and
complex 3D representations. We introduce Geometry Image Diffusion
(GIMDiffusion), a novel Text-to-3D model that utilizes geometry images to
efficiently represent 3D shapes using 2D images, thereby avoiding the need for
complex 3D-aware architectures. By integrating a Collaborative Control
mechanism, we exploit the rich 2D priors of existing Text-to-Image models such
as Stable Diffusion. This enables strong generalization even with limited 3D
training data (allowing us to use only high-quality training data) as well as
retaining compatibility with guidance techniques such as IPAdapter. In short,
GIMDiffusion enables the generation of 3D assets at speeds comparable to
current Text-to-Image models. The generated objects consist of semantically
meaningful, separate parts and include internal structures, enhancing both
usability and versatility.
66daa0964414997d108994e4
Slava Elizarov
admin_assigned
2024-09-06T09:46:38.828Z
639b56244bbcd176745a6e6d
https://cdn-avatars.huggingface.co/v1/production/uploads/639b56244bbcd176745a6e6d/a1DRanMtKUNJNcrt3ZmOg.jpeg
Slava Elizarov
user
SlavaElizarov
66daa0964414997d108994e5
Ciara Rowles
extracted_confirmed
2024-09-06T06:38:55.482Z
63357214eb6132ca653020e7
https://cdn-avatars.huggingface.co/v1/production/uploads/63357214eb6132ca653020e7/A_imoyFDx30wgi0guG_s8.png
Ciara
user
CiaraRowles
66daa0964414997d108994e6
Simon Donné
extracted_pending
2024-09-06T06:26:36.560Z
634d06167f1e86614c0e8e1e
/avatars/c1d119f42f35e0e2d7e1a655d193478b.svg
Simon Donné
user
simondonn
Geometry Image Diffusion: Fast and Data-Efficient Text-to-3D with
  Image-Based Surface Representation
Generating high-quality 3D objects from textual descriptions remains a
challenging problem due to computational cost, the scarcity of 3D data, and
complex 3D representations. We introduce Geometry Image Diffusion
(GIMDiffusion), a novel Text-to-3D model that utilizes geometry images to
efficiently represent 3D shapes using 2D images, thereby avoiding the need for
complex 3D-aware architectures. By integrating a Collaborative Control
mechanism, we exploit the rich 2D priors of existing Text-to-Image models such
as Stable Diffusion. This enables strong generalization even with limited 3D
training data (allowing us to use only high-quality training data) as well as
retaining compatibility with guidance techniques such as IPAdapter. In short,
GIMDiffusion enables the generation of 3D assets at speeds comparable to
current Text-to-Image models. The generated objects consist of semantically
meaningful, separate parts and include internal structures, enhancing both
usability and versatility.
Geometry Image Diffusion (GIMDiffusion) Text-to-3D model geometry images 3D shapes 2D images 3D-aware architectures Collaborative Control mechanism Text-to-Image models Stable Diffusion 2D priors IPAdapter 3D training data semantic separate parts internal structures