SV4D: Dynamic 3D Content Generation with Multi-Frame and Multi-View
  Consistency
We present Stable Video 4D (SV4D), a latent video diffusion model for
multi-frame and multi-view consistent dynamic 3D content generation. Unlike
previous methods that rely on separately trained generative models for video
generation and novel view synthesis, we design a unified diffusion model to
generate novel view videos of dynamic 3D objects. Specifically, given a
monocular reference video, SV4D generates novel views for each video frame that
are temporally consistent. We then use the generated novel view videos to
optimize an implicit 4D representation (dynamic NeRF) efficiently, without the
need for cumbersome SDS-based optimization used in most prior works. To train
our unified novel view video generation model, we curated a dynamic 3D object
dataset from the existing Objaverse dataset. Extensive experimental results on
multiple datasets and user studies demonstrate SV4D's state-of-the-art
performance on novel-view video synthesis as well as 4D generation compared to
prior works.
66a1af9f18ce1e6e43377dd1
Yiming Xie
admin_assigned
2024-07-25T12:23:48.432Z
65c3e6dbe61e74041888f0da
/avatars/084f51bfc9ead7ebd702e5b53d066f32.svg
Yiming Xie
user
ymxie
66a1af9f18ce1e6e43377dd2
Chun-Han Yao
admin_assigned
2024-07-25T12:24:07.251Z
659c2d500868046a55e8fa56
/avatars/b90d8af7addc0bb3360a22fd14ebfac1.svg
Chun-Han Yao
user
chunhanyao
66a1af9f18ce1e6e43377dd3
Vikram Voleti
admin_assigned
2024-07-25T12:24:12.672Z
631b8e8c8de8e645af6ec059
/avatars/75eec5fe0e8c18d7974517bf2c8bb511.svg
Vikram Voleti
user
vikky2904
66a1af9f18ce1e6e43377dd4
Huaizu Jiang
66a1af9f18ce1e6e43377dd5
Varun Jampani
admin_assigned
2024-07-25T12:24:33.345Z
630e7cfe3fc17ffc50f6e835
/avatars/0b742ff094a09f9374fafcd97ab9e002.svg
Varun Jampani
user
varunjampani
SV4D: Dynamic 3D Content Generation with Multi-Frame and Multi-View
  Consistency
We present Stable Video 4D (SV4D), a latent video diffusion model for
multi-frame and multi-view consistent dynamic 3D content generation. Unlike
previous methods that rely on separately trained generative models for video
generation and novel view synthesis, we design a unified diffusion model to
generate novel view videos of dynamic 3D objects. Specifically, given a
monocular reference video, SV4D generates novel views for each video frame that
are temporally consistent. We then use the generated novel view videos to
optimize an implicit 4D representation (dynamic NeRF) efficiently, without the
need for cumbersome SDS-based optimization used in most prior works. To train
our unified novel view video generation model, we curated a dynamic 3D object
dataset from the existing Objaverse dataset. Extensive experimental results on
multiple datasets and user studies demonstrate SV4D's state-of-the-art
performance on novel-view video synthesis as well as 4D generation compared to
prior works.
Stable Video 4D generates temporally consistent novel views for dynamic 3D objects using a unified latent video diffusion model, outperforming prior methods in both novel-view video synthesis and 4D generation.
latent video diffusion model multi-frame multi-view dynamic 3D content generation generative models video generation novel view synthesis temporally consistent implicit 4D representation dynamic NeRF SDS-based optimization Objaverse dataset