GS-LRM: Large Reconstruction Model for 3D Gaussian Splatting
We propose GS-LRM, a scalable large reconstruction model that can predict
high-quality 3D Gaussian primitives from 2-4 posed sparse images in 0.23
seconds on single A100 GPU. Our model features a very simple transformer-based
architecture; we patchify input posed images, pass the concatenated multi-view
image tokens through a sequence of transformer blocks, and decode final
per-pixel Gaussian parameters directly from these tokens for differentiable
rendering. In contrast to previous LRMs that can only reconstruct objects, by
predicting per-pixel Gaussians, GS-LRM naturally handles scenes with large
variations in scale and complexity. We show that our model can work on both
object and scene captures by training it on Objaverse and RealEstate10K
respectively. In both scenarios, the models outperform state-of-the-art
baselines by a wide margin. We also demonstrate applications of our model in
downstream 3D generation tasks. Our project webpage is available at:
https://sai-bi.github.io/project/gs-lrm/ .
6631a053472898cdc32742c7
Kai Zhang
6631a053472898cdc32742c8
Sai Bi
admin_assigned
2024-05-02T08:25:47.138Z
66303560597dc7c535168f07
/avatars/a69d52a4a198c6c8451c1d137031bf07.svg
Sai Bi
user
saibi
6631a053472898cdc32742c9
Hao Tan
6631a053472898cdc32742ca
Yuanbo Xiangli
admin_assigned
2024-05-02T08:25:26.072Z
651edd0faa888068acb5d680
/avatars/d5e03e8bc1d22bc67a3e684b52dc5598.svg
Yuanbo Xiangli
user
amberkk
6631a053472898cdc32742cb
Nanxuan Zhao
6631a053472898cdc32742cc
Kalyan Sunkavalli
6631a053472898cdc32742cd
Zexiang Xu
admin_assigned
2024-05-02T08:25:08.627Z
65565b1b8616d8d740abb976
/avatars/c245b19ed05292bf1bf4a9d8cf2a409c.svg
Zexiang Xu
user
zexiangxu
GS-LRM: Large Reconstruction Model for 3D Gaussian Splatting
We propose GS-LRM, a scalable large reconstruction model that can predict
high-quality 3D Gaussian primitives from 2-4 posed sparse images in 0.23
seconds on single A100 GPU. Our model features a very simple transformer-based
architecture; we patchify input posed images, pass the concatenated multi-view
image tokens through a sequence of transformer blocks, and decode final
per-pixel Gaussian parameters directly from these tokens for differentiable
rendering. In contrast to previous LRMs that can only reconstruct objects, by
predicting per-pixel Gaussians, GS-LRM naturally handles scenes with large
variations in scale and complexity. We show that our model can work on both
object and scene captures by training it on Objaverse and RealEstate10K
respectively. In both scenarios, the models outperform state-of-the-art
baselines by a wide margin. We also demonstrate applications of our model in
downstream 3D generation tasks. Our project webpage is available at:
https://sai-bi.github.io/project/gs-lrm/ .
GS-LRM is a scalable transformer-based model that predicts high-quality 3D Gaussian primitives from sparse images, handling both objects and scenes effectively.
transformer-based architecture patchify image tokens transformer blocks per-pixel Gaussian parameters differentiable rendering GS-LRM Objaverse RealEstate10K 3D Gaussian primitives