AutoDecoding Latent 3D Diffusion Models
We present a novel approach to the generation of static and articulated 3D
assets that has a 3D autodecoder at its core. The 3D autodecoder framework
embeds properties learned from the target dataset in the latent space, which
can then be decoded into a volumetric representation for rendering
view-consistent appearance and geometry. We then identify the appropriate
intermediate volumetric latent space, and introduce robust normalization and
de-normalization operations to learn a 3D diffusion from 2D images or monocular
videos of rigid or articulated objects. Our approach is flexible enough to use
either existing camera supervision or no camera information at all -- instead
efficiently learning it during training. Our evaluations demonstrate that our
generation results outperform state-of-the-art alternatives on various
benchmark datasets and metrics, including multi-view image datasets of
synthetic objects, real in-the-wild videos of moving people, and a large-scale,
real video dataset of static objects.
64ae08835d48838462015149
Evangelos Ntavelis
claimed_verified
2023-07-13T08:32:42.642Z
64aeca55c693a1954796e404
/avatars/4d18e907447c2370501e8e3e4088a776.svg
Evan Ntavelis
user
entavelis
64ae08835d4883846201514a
Aliaksandr Siarohin
extracted_pending
2023-07-12T07:31:58.137Z
64276311eb9a0ed86180715b
/avatars/76f933cd549f10e5e2db379de235d304.svg
Aliaksandr Siarohin
user
aliaksandr-siarohin
64ae08835d4883846201514b
Kyle Olszewski
extracted_confirmed
2023-07-12T04:48:28.327Z
63ccd965336d9d5617164be0
/avatars/21be405bdbe85d610376520180f7bb41.svg
Kyle Olszewski
user
kolsz
64ae08835d4883846201514c
Chaoyang Wang
64ae08835d4883846201514d
Luc Van Gool
64ae08835d4883846201514e
Sergey Tulyakov
AutoDecoding Latent 3D Diffusion Models
We present a novel approach to the generation of static and articulated 3D
assets that has a 3D autodecoder at its core. The 3D autodecoder framework
embeds properties learned from the target dataset in the latent space, which
can then be decoded into a volumetric representation for rendering
view-consistent appearance and geometry. We then identify the appropriate
intermediate volumetric latent space, and introduce robust normalization and
de-normalization operations to learn a 3D diffusion from 2D images or monocular
videos of rigid or articulated objects. Our approach is flexible enough to use
either existing camera supervision or no camera information at all -- instead
efficiently learning it during training. Our evaluations demonstrate that our
generation results outperform state-of-the-art alternatives on various
benchmark datasets and metrics, including multi-view image datasets of
synthetic objects, real in-the-wild videos of moving people, and a large-scale,
real video dataset of static objects.
A 3D autodecoder framework generates high-quality static and articulated 3D assets from 2D images or monocular videos, using latent space embeddings, normalization, and 3D diffusion.
3D autodecoder latent space volumetric representation robust normalization de-normalization 3D diffusion multi-view image datasets real in-the-wild videos large-scale real video dataset