ARTDECO: Towards Efficient and High-Fidelity On-the-Fly 3D
  Reconstruction with Structured Scene Representation
On-the-fly 3D reconstruction from monocular image sequences is a
long-standing challenge in computer vision, critical for applications such as
real-to-sim, AR/VR, and robotics. Existing methods face a major tradeoff:
per-scene optimization yields high fidelity but is computationally expensive,
whereas feed-forward foundation models enable real-time inference but struggle
with accuracy and robustness. In this work, we propose ARTDECO, a unified
framework that combines the efficiency of feed-forward models with the
reliability of SLAM-based pipelines. ARTDECO uses 3D foundation models for pose
estimation and point prediction, coupled with a Gaussian decoder that
transforms multi-scale features into structured 3D Gaussians. To sustain both
fidelity and efficiency at scale, we design a hierarchical Gaussian
representation with a LoD-aware rendering strategy, which improves rendering
fidelity while reducing redundancy. Experiments on eight diverse indoor and
outdoor benchmarks show that ARTDECO delivers interactive performance
comparable to SLAM, robustness similar to feed-forward systems, and
reconstruction quality close to per-scene optimization, providing a practical
path toward on-the-fly digitization of real-world environments with both
accurate geometry and high visual fidelity. Explore more demos on our project
page: https://city-super.github.io/artdeco/.
68e87dd895e8e6771df38a51
Guanghao Li
68e87dd895e8e6771df38a52
Kerui Ren
claimed_verified
2025-10-11T13:48:04.341Z
643f7178ec817b766686daed
/avatars/08b15d594a524662bf451fefa35525e2.svg
Kerui Ren
user
cskrren
68e87dd895e8e6771df38a53
Linning Xu
68e87dd895e8e6771df38a54
Zhewen Zheng
68e87dd895e8e6771df38a55
Changjian Jiang
68e87dd895e8e6771df38a56
Xin Gao
68e87dd895e8e6771df38a57
Bo Dai
68e87dd895e8e6771df38a58
Jian Pu
68e87dd895e8e6771df38a59
Mulin Yu
68e87dd895e8e6771df38a5a
Jiangmiao Pang
ARTDECO: Towards Efficient and High-Fidelity On-the-Fly 3D
  Reconstruction with Structured Scene Representation
On-the-fly 3D reconstruction from monocular image sequences is a
long-standing challenge in computer vision, critical for applications such as
real-to-sim, AR/VR, and robotics. Existing methods face a major tradeoff:
per-scene optimization yields high fidelity but is computationally expensive,
whereas feed-forward foundation models enable real-time inference but struggle
with accuracy and robustness. In this work, we propose ARTDECO, a unified
framework that combines the efficiency of feed-forward models with the
reliability of SLAM-based pipelines. ARTDECO uses 3D foundation models for pose
estimation and point prediction, coupled with a Gaussian decoder that
transforms multi-scale features into structured 3D Gaussians. To sustain both
fidelity and efficiency at scale, we design a hierarchical Gaussian
representation with a LoD-aware rendering strategy, which improves rendering
fidelity while reducing redundancy. Experiments on eight diverse indoor and
outdoor benchmarks show that ARTDECO delivers interactive performance
comparable to SLAM, robustness similar to feed-forward systems, and
reconstruction quality close to per-scene optimization, providing a practical
path toward on-the-fly digitization of real-world environments with both
accurate geometry and high visual fidelity. Explore more demos on our project
page: https://city-super.github.io/artdeco/.
ARTDECO combines feed-forward models and SLAM pipelines for efficient and accurate 3D reconstruction from monocular images.
3D reconstruction monocular image sequences real-to-sim AR/VR robotics per-scene optimization feed-forward foundation models SLAM-based pipelines 3D foundation models pose estimation point prediction Gaussian decoder multi-scale features structured 3D Gaussians hierarchical Gaussian representation LoD-aware rendering strategy rendering fidelity redundancy interactive performance reconstruction quality