ShapeGen4D: Towards High Quality 4D Shape Generation from Videos
Video-conditioned 4D shape generation aims to recover time-varying 3D
geometry and view-consistent appearance directly from an input video. In this
work, we introduce a native video-to-4D shape generation framework that
synthesizes a single dynamic 3D representation end-to-end from the video. Our
framework introduces three key components based on large-scale pre-trained 3D
models: (i) a temporal attention that conditions generation on all frames while
producing a time-indexed dynamic representation; (ii) a time-aware point
sampling and 4D latent anchoring that promote temporally consistent geometry
and texture; and (iii) noise sharing across frames to enhance temporal
stability. Our method accurately captures non-rigid motion, volume changes, and
even topological transitions without per-frame optimization. Across diverse
in-the-wild videos, our method improves robustness and perceptual fidelity and
reduces failure modes compared with the baselines.
68e654fc975ac4c405ef2275
Jiraphon Yenphraphai
claimed_verified
2025-10-09T06:51:35.786Z
63daa44df03c3d71ef33da2d
/avatars/3193967d40b82a7d678b58a8e4d0ec1a.svg
Jiraphon Yenphraphai
user
domejiraphon
68e654fc975ac4c405ef2276
Ashkan Mirzaei
68e654fc975ac4c405ef2277
Jianqi Chen
68e654fc975ac4c405ef2278
Jiaxu Zou
68e654fc975ac4c405ef2279
Sergey Tulyakov
68e654fc975ac4c405ef227a
Raymond A. Yeh
68e654fc975ac4c405ef227b
Peter Wonka
68e654fc975ac4c405ef227c
Chaoyang Wang
ShapeGen4D: Towards High Quality 4D Shape Generation from Videos
Video-conditioned 4D shape generation aims to recover time-varying 3D
geometry and view-consistent appearance directly from an input video. In this
work, we introduce a native video-to-4D shape generation framework that
synthesizes a single dynamic 3D representation end-to-end from the video. Our
framework introduces three key components based on large-scale pre-trained 3D
models: (i) a temporal attention that conditions generation on all frames while
producing a time-indexed dynamic representation; (ii) a time-aware point
sampling and 4D latent anchoring that promote temporally consistent geometry
and texture; and (iii) noise sharing across frames to enhance temporal
stability. Our method accurately captures non-rigid motion, volume changes, and
even topological transitions without per-frame optimization. Across diverse
in-the-wild videos, our method improves robustness and perceptual fidelity and
reduces failure modes compared with the baselines.
A video-to-4D shape generation framework uses temporal attention, time-aware point sampling, and noise sharing to produce dynamic 3D representations from videos, enhancing temporal stability and perceptual fidelity.
temporal attention time-aware point sampling 4D latent anchoring noise sharing non-rigid motion volume changes topological transitions dynamic 3D representation