Chupa: Carving 3D Clothed Humans from Skinned Shape Priors using 2D
  Diffusion Probabilistic Models
We propose a 3D generation pipeline that uses diffusion models to generate
realistic human digital avatars. Due to the wide variety of human identities,
poses, and stochastic details, the generation of 3D human meshes has been a
challenging problem. To address this, we decompose the problem into 2D normal
map generation and normal map-based 3D reconstruction. Specifically, we first
simultaneously generate realistic normal maps for the front and backside of a
clothed human, dubbed dual normal maps, using a pose-conditional diffusion
model. For 3D reconstruction, we ``carve'' the prior SMPL-X mesh to a detailed
3D mesh according to the normal maps through mesh optimization. To further
enhance the high-frequency details, we present a diffusion resampling scheme on
both body and facial regions, thus encouraging the generation of realistic
digital avatars. We also seamlessly incorporate a recent text-to-image
diffusion model to support text-based human identity control. Our method,
namely, Chupa, is capable of generating realistic 3D clothed humans with better
perceptual quality and identity variety.
646ac4954c1cd18b49798325
Byungjun Kim
claimed_verified
2023-05-22T11:57:29.675Z
644f822623d7eb05ca691d95
/avatars/b9d3f39155ee96a79b8c873e4bab57e6.svg
Byungjun Kim
user
byungjun-kim
646ac4954c1cd18b49798326
Patrick Kwon
claimed_verified
2023-05-22T11:57:26.952Z
63044b2e1dd5d3c624882d19
/avatars/ba4d387547d1d0baeca918caea680f89.svg
Patrick Kwon
user
yj7082126
646ac4954c1cd18b49798327
Kwangho Lee
646ac4954c1cd18b49798328
Myunggi Lee
646ac4954c1cd18b49798329
Sookwan Han
claimed_verified
2023-09-27T10:41:01.594Z
628667494713da0266f4d998
/avatars/64736ed1ad48ee75e3cf53e78b4b622b.svg
Sookwan Han
user
jellyheadnadrew
646ac4954c1cd18b4979832a
Daesik Kim
646ac4954c1cd18b4979832b
Hanbyul Joo
claimed_verified
2024-01-24T12:19:09.513Z
65b0ebe1165e69e0e6dc6ddd
/avatars/d2e95b2040c761200df1293db2949da7.svg
Hanbyul Joo
user
jhugestar
Chupa: Carving 3D Clothed Humans from Skinned Shape Priors using 2D
  Diffusion Probabilistic Models
We propose a 3D generation pipeline that uses diffusion models to generate
realistic human digital avatars. Due to the wide variety of human identities,
poses, and stochastic details, the generation of 3D human meshes has been a
challenging problem. To address this, we decompose the problem into 2D normal
map generation and normal map-based 3D reconstruction. Specifically, we first
simultaneously generate realistic normal maps for the front and backside of a
clothed human, dubbed dual normal maps, using a pose-conditional diffusion
model. For 3D reconstruction, we ``carve'' the prior SMPL-X mesh to a detailed
3D mesh according to the normal maps through mesh optimization. To further
enhance the high-frequency details, we present a diffusion resampling scheme on
both body and facial regions, thus encouraging the generation of realistic
digital avatars. We also seamlessly incorporate a recent text-to-image
diffusion model to support text-based human identity control. Our method,
namely, Chupa, is capable of generating realistic 3D clothed humans with better
perceptual quality and identity variety.
A 3D avatar generation pipeline utilizes diffusion models for dual normal map creation and 3D reconstruction, enhanced with text-based identity control.
diffusion models 3D human meshes 2D normal map generation normal map-based 3D reconstruction dual normal maps pose-conditional diffusion model SMPL-X mesh mesh optimization diffusion resampling scheme text-to-image diffusion model