4DNeX: Feed-Forward 4D Generative Modeling Made Easy
We present 4DNeX, the first feed-forward framework for generating 4D (i.e.,
dynamic 3D) scene representations from a single image. In contrast to existing
methods that rely on computationally intensive optimization or require
multi-frame video inputs, 4DNeX enables efficient, end-to-end image-to-4D
generation by fine-tuning a pretrained video diffusion model. Specifically, 1)
to alleviate the scarcity of 4D data, we construct 4DNeX-10M, a large-scale
dataset with high-quality 4D annotations generated using advanced
reconstruction approaches. 2) we introduce a unified 6D video representation
that jointly models RGB and XYZ sequences, facilitating structured learning of
both appearance and geometry. 3) we propose a set of simple yet effective
adaptation strategies to repurpose pretrained video diffusion models for 4D
modeling. 4DNeX produces high-quality dynamic point clouds that enable
novel-view video synthesis. Extensive experiments demonstrate that 4DNeX
outperforms existing 4D generation methods in efficiency and generalizability,
offering a scalable solution for image-to-4D modeling and laying the foundation
for generative 4D world models that simulate dynamic scene evolution.
68a3dfefb65388761d074471
Zhaoxi Chen
68a3dfefb65388761d074472
Tianqi Liu
68a3dfefb65388761d074473
Long Zhuo
68a3dfefb65388761d074474
Jiawei Ren
68a3dfefb65388761d074475
Zeng Tao
68a3dfefb65388761d074476
He Zhu
68a3dfefb65388761d074477
Fangzhou Hong
68a3dfefb65388761d074478
Liang Pan
68a3dfefb65388761d074479
Ziwei Liu
4DNeX: Feed-Forward 4D Generative Modeling Made Easy
We present 4DNeX, the first feed-forward framework for generating 4D (i.e.,
dynamic 3D) scene representations from a single image. In contrast to existing
methods that rely on computationally intensive optimization or require
multi-frame video inputs, 4DNeX enables efficient, end-to-end image-to-4D
generation by fine-tuning a pretrained video diffusion model. Specifically, 1)
to alleviate the scarcity of 4D data, we construct 4DNeX-10M, a large-scale
dataset with high-quality 4D annotations generated using advanced
reconstruction approaches. 2) we introduce a unified 6D video representation
that jointly models RGB and XYZ sequences, facilitating structured learning of
both appearance and geometry. 3) we propose a set of simple yet effective
adaptation strategies to repurpose pretrained video diffusion models for 4D
modeling. 4DNeX produces high-quality dynamic point clouds that enable
novel-view video synthesis. Extensive experiments demonstrate that 4DNeX
outperforms existing 4D generation methods in efficiency and generalizability,
offering a scalable solution for image-to-4D modeling and laying the foundation
for generative 4D world models that simulate dynamic scene evolution.
4DNeX generates high-quality dynamic 3D scene representations from a single image using a fine-tuned pretrained video diffusion model, outperforming existing methods in efficiency and generalizability.
feed-forward framework 4D scene representations video diffusion model 4DNeX-10M 6D video representation RGB XYZ sequences dynamic point clouds novel-view video synthesis generative 4D world models