HeadCraft: Modeling High-Detail Shape Variations for Animated 3DMMs
Current advances in human head modeling allow to generate plausible-looking
3D head models via neural representations. Nevertheless, constructing complete
high-fidelity head models with explicitly controlled animation remains an
issue. Furthermore, completing the head geometry based on a partial
observation, e.g. coming from a depth sensor, while preserving details is often
problematic for the existing methods. We introduce a generative model for
detailed 3D head meshes on top of an articulated 3DMM which allows explicit
animation and high-detail preservation at the same time. Our method is trained
in two stages. First, we register a parametric head model with vertex
displacements to each mesh of the recently introduced NPHM dataset of accurate
3D head scans. The estimated displacements are baked into a hand-crafted UV
layout. Second, we train a StyleGAN model in order to generalize over the UV
maps of displacements. The decomposition of the parametric model and
high-quality vertex displacements allows us to animate the model and modify it
semantically. We demonstrate the results of unconditional generation and
fitting to the full or partial observation. The project page is available at
https://seva100.github.io/headcraft.
658506c4c76915aa14cc8bb8
Artem Sevastopolsky
claimed_verified
2023-12-22T14:28:59.317Z
64638395d2044cd1d7c7e832
/avatars/6541363767ba7a91b59e68c400817376.svg
Artem Sevastopolskiy
user
seva100
658506c4c76915aa14cc8bb9
Philip-William Grassal
658506c4c76915aa14cc8bba
Simon Giebenhain
admin_assigned
2023-12-22T12:34:05.641Z
64dc981b96fbcd6b19f687b5
/avatars/6eca33e3fcca750d4f6e9b0f7d724d8e.svg
Giebenhain
user
SimonGiebenhain
658506c4c76915aa14cc8bbb
ShahRukh Athar
admin_assigned
2023-12-22T12:34:13.916Z
650c3a2bd439bbbbadf40436
/avatars/a5a7c04a03580ae59c28eb6f90d43b47.svg
ShahRukh Athar
user
Sathar
658506c4c76915aa14cc8bbc
Luisa Verdoliva
658506c4c76915aa14cc8bbd
Matthias Niessner
HeadCraft: Modeling High-Detail Shape Variations for Animated 3DMMs
Current advances in human head modeling allow to generate plausible-looking
3D head models via neural representations. Nevertheless, constructing complete
high-fidelity head models with explicitly controlled animation remains an
issue. Furthermore, completing the head geometry based on a partial
observation, e.g. coming from a depth sensor, while preserving details is often
problematic for the existing methods. We introduce a generative model for
detailed 3D head meshes on top of an articulated 3DMM which allows explicit
animation and high-detail preservation at the same time. Our method is trained
in two stages. First, we register a parametric head model with vertex
displacements to each mesh of the recently introduced NPHM dataset of accurate
3D head scans. The estimated displacements are baked into a hand-crafted UV
layout. Second, we train a StyleGAN model in order to generalize over the UV
maps of displacements. The decomposition of the parametric model and
high-quality vertex displacements allows us to animate the model and modify it
semantically. We demonstrate the results of unconditional generation and
fitting to the full or partial observation. The project page is available at
https://seva100.github.io/headcraft.
A generative model combining an articulated 3DMM and StyleGAN achieves detailed and animatable 3D head models from partial observations.
neural representations 3D head models articulated 3DMM StyleGAN vertex displacements UV layout unconditional generation