CAT4D: Create Anything in 4D with Multi-View Video Diffusion Models
We present CAT4D, a method for creating 4D (dynamic 3D) scenes from monocular
video. CAT4D leverages a multi-view video diffusion model trained on a diverse
combination of datasets to enable novel view synthesis at any specified camera
poses and timestamps. Combined with a novel sampling approach, this model can
transform a single monocular video into a multi-view video, enabling robust 4D
reconstruction via optimization of a deformable 3D Gaussian representation. We
demonstrate competitive performance on novel view synthesis and dynamic scene
reconstruction benchmarks, and highlight the creative capabilities for 4D scene
generation from real or generated videos. See our project page for results and
interactive demos: cat-4d.github.io.
6747e4ce2e5e8d6c8522f6f7
Rundi Wu
claimed_verified
2024-11-28T09:19:38.809Z
63823d59a18d1ca09653aa21
/avatars/151e1ca46627abdd99c27945ea2fa441.svg
Rundi Wu
user
rundi
6747e4ce2e5e8d6c8522f6f8
Ruiqi Gao
6747e4ce2e5e8d6c8522f6f9
Ben Poole
admin_assigned
2024-11-28T10:35:38.276Z
63213768d2d45f315183abc3
/avatars/521f52f430b73e514afb06e3a9d6779d.svg
Ben Poole
user
doinkda
6747e4ce2e5e8d6c8522f6fa
Alex Trevithick
admin_assigned
2024-11-28T10:35:32.163Z
656a76839dcedd16d5d4ed49
/avatars/81e4a17f9af8c990dc13bf1a7bb4efbd.svg
Alex Trevithick
user
atrevithick
6747e4ce2e5e8d6c8522f6fb
Changxi Zheng
6747e4ce2e5e8d6c8522f6fc
Jonathan T. Barron
6747e4ce2e5e8d6c8522f6fd
Aleksander Holynski
admin_assigned
2024-11-28T10:35:58.104Z
62f6e2792e53c2efd33faa92
/avatars/85d94b4022577747b8d2d10a82c2f3c7.svg
Aleksander Holynski
user
holynski
CAT4D: Create Anything in 4D with Multi-View Video Diffusion Models
We present CAT4D, a method for creating 4D (dynamic 3D) scenes from monocular
video. CAT4D leverages a multi-view video diffusion model trained on a diverse
combination of datasets to enable novel view synthesis at any specified camera
poses and timestamps. Combined with a novel sampling approach, this model can
transform a single monocular video into a multi-view video, enabling robust 4D
reconstruction via optimization of a deformable 3D Gaussian representation. We
demonstrate competitive performance on novel view synthesis and dynamic scene
reconstruction benchmarks, and highlight the creative capabilities for 4D scene
generation from real or generated videos. See our project page for results and
interactive demos: cat-4d.github.io.
CAT4D is a method that uses a multi-view video diffusion model to generate 4D dynamic scenes from monocular video by optimizing a deformable 3D Gaussian representation.
multi-view video diffusion model novel view synthesis dynamic scene reconstruction deformable 3D Gaussian representation