TTT3R: 3D Reconstruction as Test-Time Training
Modern Recurrent Neural Networks have become a competitive architecture for
3D reconstruction due to their linear-time complexity. However, their
performance degrades significantly when applied beyond the training context
length, revealing limited length generalization. In this work, we revisit the
3D reconstruction foundation models from a Test-Time Training perspective,
framing their designs as an online learning problem. Building on this
perspective, we leverage the alignment confidence between the memory state and
incoming observations to derive a closed-form learning rate for memory updates,
to balance between retaining historical information and adapting to new
observations. This training-free intervention, termed TTT3R, substantially
improves length generalization, achieving a 2times improvement in global
pose estimation over baselines, while operating at 20 FPS with just 6 GB of GPU
memory to process thousands of images. Code available in
https://rover-xingyu.github.io/TTT3R
68dcd6a04159d1f2418f9c71
Xingyu Chen
claimed_verified
2025-10-01T10:20:41.482Z
66606a13fc6c0816442bd161
https://cdn-avatars.huggingface.co/v1/production/uploads/66606a13fc6c0816442bd161/tS8pBDXEb3QkIjvZao55l.jpeg
Xingyu Chen
user
rover-xingyu
68dcd6a04159d1f2418f9c72
Yue Chen
claimed_verified
2025-10-06T03:21:25.176Z
66f80281d88dc2ad510663e9
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/5HV3mTu-cxPCnWlCi_2wB.jpeg
Yue Chen
user
faneggg
68dcd6a04159d1f2418f9c73
Yuliang Xiu
claimed_verified
2025-10-09T06:52:18.354Z
6183d0b249ef1d984699e4a3
https://cdn-avatars.huggingface.co/v1/production/uploads/6183d0b249ef1d984699e4a3/M6B9b73v8su9H8jfdcbgu.jpeg
Yuliang Xiu
user
Yuliang
68dcd6a04159d1f2418f9c74
Andreas Geiger
68dcd6a04159d1f2418f9c75
Anpei Chen
TTT3R: 3D Reconstruction as Test-Time Training
Modern Recurrent Neural Networks have become a competitive architecture for
3D reconstruction due to their linear-time complexity. However, their
performance degrades significantly when applied beyond the training context
length, revealing limited length generalization. In this work, we revisit the
3D reconstruction foundation models from a Test-Time Training perspective,
framing their designs as an online learning problem. Building on this
perspective, we leverage the alignment confidence between the memory state and
incoming observations to derive a closed-form learning rate for memory updates,
to balance between retaining historical information and adapting to new
observations. This training-free intervention, termed TTT3R, substantially
improves length generalization, achieving a 2times improvement in global
pose estimation over baselines, while operating at 20 FPS with just 6 GB of GPU
memory to process thousands of images. Code available in
https://rover-xingyu.github.io/TTT3R
TTT3R, a test-time training intervention, enhances length generalization in 3D reconstruction by dynamically adjusting memory updates based on alignment confidence, improving global pose estimation and processing efficiency.
Recurrent Neural Networks 3D reconstruction linear-time complexity length generalization Test-Time Training memory state incoming observations closed-form learning rate global pose estimation FPS GPU memory