SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image
  using Latent Video Diffusion
We present Stable Video 3D (SV3D) -- a latent video diffusion model for
high-resolution, image-to-multi-view generation of orbital videos around a 3D
object. Recent work on 3D generation propose techniques to adapt 2D generative
models for novel view synthesis (NVS) and 3D optimization. However, these
methods have several disadvantages due to either limited views or inconsistent
NVS, thereby affecting the performance of 3D object generation. In this work,
we propose SV3D that adapts image-to-video diffusion model for novel multi-view
synthesis and 3D generation, thereby leveraging the generalization and
multi-view consistency of the video models, while further adding explicit
camera control for NVS. We also propose improved 3D optimization techniques to
use SV3D and its NVS outputs for image-to-3D generation. Extensive experimental
results on multiple datasets with 2D and 3D metrics as well as user study
demonstrate SV3D's state-of-the-art performance on NVS as well as 3D
reconstruction compared to prior works.
65f90aa244d5f5e014ad1eee
Vikram Voleti
admin_assigned
2024-03-19T09:39:15.493Z
631b8e8c8de8e645af6ec059
/avatars/75eec5fe0e8c18d7974517bf2c8bb511.svg
Vikram Voleti
user
vikky2904
65f90aa244d5f5e014ad1eef
Chun-Han Yao
admin_assigned
2024-03-19T09:39:28.275Z
659c2d500868046a55e8fa56
/avatars/b90d8af7addc0bb3360a22fd14ebfac1.svg
Chun-Han Yao
user
chunhanyao
65f90aa244d5f5e014ad1ef0
Mark Boss
claimed_verified
2024-03-19T08:45:23.279Z
64b7f06fda8017900e893eb4
https://cdn-avatars.huggingface.co/v1/production/uploads/64b7f06fda8017900e893eb4/3VcQFjERXyAjHQFLMFEJt.jpeg
Mark Boss
user
mboss
65f90aa244d5f5e014ad1ef1
Adam Letts
admin_assigned
2024-03-19T09:39:35.094Z
62a43e93d83c3facaf8b81ef
/avatars/4f710707c790f88bb358992ce863b7b1.svg
Adam Letts
user
gandamu
65f90aa244d5f5e014ad1ef2
David Pankratz
65f90aa244d5f5e014ad1ef3
Dmitry Tochilkin
65f90aa244d5f5e014ad1ef4
Christian Laforte
admin_assigned
2024-03-19T09:39:51.844Z
62f59a712873214eb5f5c2c4
/avatars/1a518c1ea70d3e60eeac8f4e27b2fcb5.svg
Christian Laforte
user
claforte
65f90aa244d5f5e014ad1ef5
Robin Rombach
admin_assigned
2024-03-19T09:39:59.039Z
62cfefa74b3e8dc1e32e38bf
https://cdn-avatars.huggingface.co/v1/production/uploads/1657794439283-noauth.png
Robin Rombach
user
rromb
65f90aa244d5f5e014ad1ef6
Varun Jampani
admin_assigned
2024-03-19T09:40:05.036Z
630e7cfe3fc17ffc50f6e835
/avatars/0b742ff094a09f9374fafcd97ab9e002.svg
Varun Jampani
user
varunjampani
SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image
  using Latent Video Diffusion
We present Stable Video 3D (SV3D) -- a latent video diffusion model for
high-resolution, image-to-multi-view generation of orbital videos around a 3D
object. Recent work on 3D generation propose techniques to adapt 2D generative
models for novel view synthesis (NVS) and 3D optimization. However, these
methods have several disadvantages due to either limited views or inconsistent
NVS, thereby affecting the performance of 3D object generation. In this work,
we propose SV3D that adapts image-to-video diffusion model for novel multi-view
synthesis and 3D generation, thereby leveraging the generalization and
multi-view consistency of the video models, while further adding explicit
camera control for NVS. We also propose improved 3D optimization techniques to
use SV3D and its NVS outputs for image-to-3D generation. Extensive experimental
results on multiple datasets with 2D and 3D metrics as well as user study
demonstrate SV3D's state-of-the-art performance on NVS as well as 3D
reconstruction compared to prior works.
Stable Video 3D (SV3D) achieves high-resolution, multi-view 3D object generation by adapting image-to-video diffusion models and improving 3D optimization techniques.
latent video diffusion model high-resolution image-to-multi-view generation orbital videos novel view synthesis (NVS) 3D object generation image-to-video diffusion model multi-view consistency camera control image-to-3D generation 3D optimization techniques NVS 3D reconstruction