LRM-Zero: Training Large Reconstruction Models with Synthesized Data
We present LRM-Zero, a Large Reconstruction Model (LRM) trained entirely on
synthesized 3D data, achieving high-quality sparse-view 3D reconstruction. The
core of LRM-Zero is our procedural 3D dataset, Zeroverse, which is
automatically synthesized from simple primitive shapes with random texturing
and augmentations (e.g., height fields, boolean differences, and wireframes).
Unlike previous 3D datasets (e.g., Objaverse) which are often captured or
crafted by humans to approximate real 3D data, Zeroverse completely ignores
realistic global semantics but is rich in complex geometric and texture details
that are locally similar to or even more intricate than real objects. We
demonstrate that our LRM-Zero, trained with our fully synthesized Zeroverse,
can achieve high visual quality in the reconstruction of real-world objects,
competitive with models trained on Objaverse. We also analyze several critical
design choices of Zeroverse that contribute to LRM-Zero's capability and
training stability. Our work demonstrates that 3D reconstruction, one of the
core tasks in 3D vision, can potentially be addressed without the semantics of
real-world objects. The Zeroverse's procedural synthesis code and interactive
visualization are available at: https://desaixie.github.io/lrm-zero/.
666bd7d2f1b3867caf4b73c7
Desai Xie
admin_assigned
2024-06-14T11:09:38.093Z
6279f597812ee439d9c8de67
/avatars/49bea87294118e18df7b774f716af81a.svg
Desai Xie
user
desaix
666bd7d2f1b3867caf4b73c8
Sai Bi
admin_assigned
2024-06-14T11:09:44.136Z
66303560597dc7c535168f07
/avatars/a69d52a4a198c6c8451c1d137031bf07.svg
Sai Bi
user
saibi
666bd7d2f1b3867caf4b73c9
Zhixin Shu
admin_assigned
2024-06-14T11:09:51.363Z
62a8efd508a7ea93ff18785a
/avatars/ff259233d437833a304329bb973a5a04.svg
Zhixin Shu
user
zhixinshu
666bd7d2f1b3867caf4b73ca
Kai Zhang
admin_assigned
2024-06-14T11:10:15.966Z
63e0a50242591dda0b9dca5c
https://cdn-avatars.huggingface.co/v1/production/uploads/63e0a50242591dda0b9dca5c/c7cBPEBWQDFYimfGnO_SI.png
Kai Zhang
user
drogozhang
666bd7d2f1b3867caf4b73cb
Zexiang Xu
admin_assigned
2024-06-14T11:10:21.402Z
65565b1b8616d8d740abb976
/avatars/c245b19ed05292bf1bf4a9d8cf2a409c.svg
Zexiang Xu
user
zexiangxu
666bd7d2f1b3867caf4b73cc
Yi Zhou
admin_assigned
2024-06-14T11:11:08.044Z
6621dc68486de1d8db526535
/avatars/1c8d6a62e6e61fda98748c2656e17851.svg
Yi Zhou
user
yizhou-seu
666bd7d2f1b3867caf4b73cd
Sören Pirk
extracted_confirmed
2024-06-14T05:56:40.690Z
658e87be43edad2169db050a
/avatars/467d8949626539a7482dccbee4fb4d25.svg
Soren Pirk
user
pirk
666bd7d2f1b3867caf4b73ce
Arie Kaufman
666bd7d2f1b3867caf4b73cf
Xin Sun
666bd7d2f1b3867caf4b73d0
Hao Tan
LRM-Zero: Training Large Reconstruction Models with Synthesized Data
We present LRM-Zero, a Large Reconstruction Model (LRM) trained entirely on
synthesized 3D data, achieving high-quality sparse-view 3D reconstruction. The
core of LRM-Zero is our procedural 3D dataset, Zeroverse, which is
automatically synthesized from simple primitive shapes with random texturing
and augmentations (e.g., height fields, boolean differences, and wireframes).
Unlike previous 3D datasets (e.g., Objaverse) which are often captured or
crafted by humans to approximate real 3D data, Zeroverse completely ignores
realistic global semantics but is rich in complex geometric and texture details
that are locally similar to or even more intricate than real objects. We
demonstrate that our LRM-Zero, trained with our fully synthesized Zeroverse,
can achieve high visual quality in the reconstruction of real-world objects,
competitive with models trained on Objaverse. We also analyze several critical
design choices of Zeroverse that contribute to LRM-Zero's capability and
training stability. Our work demonstrates that 3D reconstruction, one of the
core tasks in 3D vision, can potentially be addressed without the semantics of
real-world objects. The Zeroverse's procedural synthesis code and interactive
visualization are available at: https://desaixie.github.io/lrm-zero/.
LRM-Zero, a 3D reconstruction model trained on synthesized data, achieves high visual quality comparable to models using real-world datasets.
Large Reconstruction Model 3D reconstruction procedural 3D dataset Zeroverse primitive shapes random texturing augmentations height fields boolean differences wireframes Objaverse 3D vision semantic semantics procedural synthesis