MVDiffusion++: A Dense High-resolution Multi-view Diffusion Model for
  Single or Sparse-view 3D Object Reconstruction
This paper presents a neural architecture MVDiffusion++ for 3D object
reconstruction that synthesizes dense and high-resolution views of an object
given one or a few images without camera poses. MVDiffusion++ achieves superior
flexibility and scalability with two surprisingly simple ideas: 1) A
``pose-free architecture'' where standard self-attention among 2D latent
features learns 3D consistency across an arbitrary number of conditional and
generation views without explicitly using camera pose information; and 2) A
``view dropout strategy'' that discards a substantial number of output views
during training, which reduces the training-time memory footprint and enables
dense and high-resolution view synthesis at test time. We use the Objaverse for
training and the Google Scanned Objects for evaluation with standard novel view
synthesis and 3D reconstruction metrics, where MVDiffusion++ significantly
outperforms the current state of the arts. We also demonstrate a text-to-3D
application example by combining MVDiffusion++ with a text-to-image generative
model.
65d56914a71cb46f50cfc21a
Shitao Tang
65d56914a71cb46f50cfc21b
Jiacheng Chen
claimed_verified
2024-10-04T07:32:36.151Z
655bca95360e4f90cb61ba83
/avatars/1a187beb91a5e2fdc2303620b742aab1.svg
Jiacheng Chen
user
cccjc
65d56914a71cb46f50cfc21c
Dilin Wang
65d56914a71cb46f50cfc21d
Chengzhou Tang
65d56914a71cb46f50cfc21e
Fuyang Zhang
65d56914a71cb46f50cfc21f
Yuchen Fan
65d56914a71cb46f50cfc220
Vikas Chandra
admin_assigned
2024-02-21T10:31:48.934Z
6566681f7b5ed0735812af32
/avatars/424905abb2973954b0850c592743b6fb.svg
Vikas Chandra
user
vchandra
65d56914a71cb46f50cfc221
Yasutaka Furukawa
65d56914a71cb46f50cfc222
Rakesh Ranjan
MVDiffusion++: A Dense High-resolution Multi-view Diffusion Model for
  Single or Sparse-view 3D Object Reconstruction
This paper presents a neural architecture MVDiffusion++ for 3D object
reconstruction that synthesizes dense and high-resolution views of an object
given one or a few images without camera poses. MVDiffusion++ achieves superior
flexibility and scalability with two surprisingly simple ideas: 1) A
``pose-free architecture'' where standard self-attention among 2D latent
features learns 3D consistency across an arbitrary number of conditional and
generation views without explicitly using camera pose information; and 2) A
``view dropout strategy'' that discards a substantial number of output views
during training, which reduces the training-time memory footprint and enables
dense and high-resolution view synthesis at test time. We use the Objaverse for
training and the Google Scanned Objects for evaluation with standard novel view
synthesis and 3D reconstruction metrics, where MVDiffusion++ significantly
outperforms the current state of the arts. We also demonstrate a text-to-3D
application example by combining MVDiffusion++ with a text-to-image generative
model.
MVDiffusion++, a neural architecture for 3D object reconstruction, synthesizes high-resolution views from limited images using a pose-free self-attention mechanism and a view dropout strategy, achieving superior performance in novel view synthesis and text-to-3D applications.
pose-free architecture 2D latent features 3D consistency view dropout strategy MVDiffusion++ Objaverse Google Scanned Objects novel view synthesis 3D reconstruction text-to-image generative model