Garment3DGen: 3D Garment Stylization and Texture Generation
We introduce Garment3DGen a new method to synthesize 3D garment assets from a
base mesh given a single input image as guidance. Our proposed approach allows
users to generate 3D textured clothes based on both real and synthetic images,
such as those generated by text prompts. The generated assets can be directly
draped and simulated on human bodies. First, we leverage the recent progress of
image to 3D diffusion methods to generate 3D garment geometries. However, since
these geometries cannot be utilized directly for downstream tasks, we propose
to use them as pseudo ground-truth and set up a mesh deformation optimization
procedure that deforms a base template mesh to match the generated 3D target.
Second, we introduce carefully designed losses that allow the input base mesh
to freely deform towards the desired target, yet preserve mesh quality and
topology such that they can be simulated. Finally, a texture estimation module
generates high-fidelity texture maps that are globally and locally consistent
and faithfully capture the input guidance, allowing us to render the generated
3D assets. With Garment3DGen users can generate the textured 3D garment of
their choice without the need of artist intervention. One can provide a textual
prompt describing the garment they desire to generate a simulation-ready 3D
asset. We present a plethora of quantitative and qualitative comparisons on
various assets both real and generated and provide use-cases of how one can
generate simulation-ready 3D garments.
6604e1d41eb292d6df3856a1
Nikolaos Sarafianos
6604e1d41eb292d6df3856a2
Tuur Stuyck
6604e1d41eb292d6df3856a3
Xiaoyu Xiang
6604e1d41eb292d6df3856a4
Yilei Li
6604e1d41eb292d6df3856a5
Jovan Popovic
6604e1d41eb292d6df3856a6
Rakesh Ranjan
Garment3DGen: 3D Garment Stylization and Texture Generation
We introduce Garment3DGen a new method to synthesize 3D garment assets from a
base mesh given a single input image as guidance. Our proposed approach allows
users to generate 3D textured clothes based on both real and synthetic images,
such as those generated by text prompts. The generated assets can be directly
draped and simulated on human bodies. First, we leverage the recent progress of
image to 3D diffusion methods to generate 3D garment geometries. However, since
these geometries cannot be utilized directly for downstream tasks, we propose
to use them as pseudo ground-truth and set up a mesh deformation optimization
procedure that deforms a base template mesh to match the generated 3D target.
Second, we introduce carefully designed losses that allow the input base mesh
to freely deform towards the desired target, yet preserve mesh quality and
topology such that they can be simulated. Finally, a texture estimation module
generates high-fidelity texture maps that are globally and locally consistent
and faithfully capture the input guidance, allowing us to render the generated
3D assets. With Garment3DGen users can generate the textured 3D garment of
their choice without the need of artist intervention. One can provide a textual
prompt describing the garment they desire to generate a simulation-ready 3D
asset. We present a plethora of quantitative and qualitative comparisons on
various assets both real and generated and provide use-cases of how one can
generate simulation-ready 3D garments.
Garment3DGen synthesizes 3D textile assets from 2D images using diffusion methods and mesh deformation, enabling simulation-ready garments from real or generated images, including textual prompts.
image to 3D diffusion methods mesh deformation optimization pseudo ground-truth mesh quality topology preservation texture estimation high-fidelity texture maps quantitative comparisons qualitative comparisons