DELTA: Dense Efficient Long-range 3D Tracking for any video
Tracking dense 3D motion from monocular videos remains challenging,
particularly when aiming for pixel-level precision over long sequences. We
introduce \Approach, a novel method that efficiently tracks every pixel in 3D
space, enabling accurate motion estimation across entire videos. Our approach
leverages a joint global-local attention mechanism for reduced-resolution
tracking, followed by a transformer-based upsampler to achieve high-resolution
predictions. Unlike existing methods, which are limited by computational
inefficiency or sparse tracking, \Approach delivers dense 3D tracking at scale,
running over 8x faster than previous methods while achieving state-of-the-art
accuracy. Furthermore, we explore the impact of depth representation on
tracking performance and identify log-depth as the optimal choice. Extensive
experiments demonstrate the superiority of \Approach on multiple benchmarks,
achieving new state-of-the-art results in both 2D and 3D dense tracking tasks.
Our method provides a robust solution for applications requiring fine-grained,
long-term motion tracking in 3D space.
67245ee3003a1dfcc5cc2577
Tuan Duc Ngo
67245ee3003a1dfcc5cc2578
Peiye Zhuang
67245ee3003a1dfcc5cc2579
Chuang Gan
67245ee3003a1dfcc5cc257a
Evangelos Kalogerakis
67245ee3003a1dfcc5cc257b
Sergey Tulyakov
67245ee3003a1dfcc5cc257c
Hsin-Ying Lee
67245ee3003a1dfcc5cc257d
Chaoyang Wang
DELTA: Dense Efficient Long-range 3D Tracking for any video
Tracking dense 3D motion from monocular videos remains challenging,
particularly when aiming for pixel-level precision over long sequences. We
introduce \Approach, a novel method that efficiently tracks every pixel in 3D
space, enabling accurate motion estimation across entire videos. Our approach
leverages a joint global-local attention mechanism for reduced-resolution
tracking, followed by a transformer-based upsampler to achieve high-resolution
predictions. Unlike existing methods, which are limited by computational
inefficiency or sparse tracking, \Approach delivers dense 3D tracking at scale,
running over 8x faster than previous methods while achieving state-of-the-art
accuracy. Furthermore, we explore the impact of depth representation on
tracking performance and identify log-depth as the optimal choice. Extensive
experiments demonstrate the superiority of \Approach on multiple benchmarks,
achieving new state-of-the-art results in both 2D and 3D dense tracking tasks.
Our method provides a robust solution for applications requiring fine-grained,
long-term motion tracking in 3D space.
A novel method achieves dense 3D motion tracking from monocular videos using a joint global-local attention mechanism and transformer-based upsampling, outperforming existing methods in accuracy and computational efficiency.
joint global-local attention mechanism transformer-based upsampler log-depth dense 3D tracking monocular videos motion estimation