GRM: Large Gaussian Reconstruction Model for Efficient 3D Reconstruction
  and Generation
We introduce GRM, a large-scale reconstructor capable of recovering a 3D
asset from sparse-view images in around 0.1s. GRM is a feed-forward
transformer-based model that efficiently incorporates multi-view information to
translate the input pixels into pixel-aligned Gaussians, which are unprojected
to create a set of densely distributed 3D Gaussians representing a scene.
Together, our transformer architecture and the use of 3D Gaussians unlock a
scalable and efficient reconstruction framework. Extensive experimental results
demonstrate the superiority of our method over alternatives regarding both
reconstruction quality and efficiency. We also showcase the potential of GRM in
generative tasks, i.e., text-to-3D and image-to-3D, by integrating it with
existing multi-view diffusion models. Our project website is at:
https://justimyhxu.github.io/projects/grm/.
65fd03a2023aaa2beed19999
Yinghao Xu
admin_assigned
2024-03-22T09:48:36.739Z
6555da9adcf410fd0753569c
/avatars/ac58a796bb54f334fdb475a4b75c4d27.svg
Yinghao Xu
user
justimyhxu
65fd03a2023aaa2beed1999a
Zifan Shi
65fd03a2023aaa2beed1999b
Wang Yifan
65fd03a2023aaa2beed1999c
Hansheng Chen
admin_assigned
2024-03-22T09:50:43.634Z
638067fcb334960c987fbeda
/avatars/87f1eaaf6b3a9c0d47d6f406261ccc18.svg
Hansheng Chen
user
Lakonik
65fd03a2023aaa2beed1999d
Ceyuan Yang
admin_assigned
2024-03-22T09:58:37.119Z
64accf3fc1da7c4dbc69392d
/avatars/ec8ad2498f8a49a7a00c77a7f70d34bb.svg
Ceyuan
user
Ceyuan
65fd03a2023aaa2beed1999e
Sida Peng
admin_assigned
2024-03-22T09:58:43.680Z
62986ca2b58e71e2ac9b8f01
/avatars/83944db5f3dbb6f47c47c46fb2cb2849.svg
Sida Peng
user
pengsida
65fd03a2023aaa2beed1999f
Yujun Shen
65fd03a2023aaa2beed199a0
Gordon Wetzstein
GRM: Large Gaussian Reconstruction Model for Efficient 3D Reconstruction
  and Generation
We introduce GRM, a large-scale reconstructor capable of recovering a 3D
asset from sparse-view images in around 0.1s. GRM is a feed-forward
transformer-based model that efficiently incorporates multi-view information to
translate the input pixels into pixel-aligned Gaussians, which are unprojected
to create a set of densely distributed 3D Gaussians representing a scene.
Together, our transformer architecture and the use of 3D Gaussians unlock a
scalable and efficient reconstruction framework. Extensive experimental results
demonstrate the superiority of our method over alternatives regarding both
reconstruction quality and efficiency. We also showcase the potential of GRM in
generative tasks, i.e., text-to-3D and image-to-3D, by integrating it with
existing multi-view diffusion models. Our project website is at:
https://justimyhxu.github.io/projects/grm/.
GRM, a transformer-based model, efficiently reconstructs 3D assets from sparse images using pixel-aligned Gaussians and achieves superior quality and speed compared to alternatives.
transformer-based model 3D Gaussians reconstruction quality efficiency text-to-3D image-to-3D multi-view diffusion models