DA^2: Depth Anything in Any Direction
Panorama has a full FoV (360^circtimes180^circ), offering a more
complete visual description than perspective images. Thanks to this
characteristic, panoramic depth estimation is gaining increasing traction in 3D
vision. However, due to the scarcity of panoramic data, previous methods are
often restricted to in-domain settings, leading to poor zero-shot
generalization. Furthermore, due to the spherical distortions inherent in
panoramas, many approaches rely on perspective splitting (e.g., cubemaps),
which leads to suboptimal efficiency. To address these challenges, we propose
DA^{2}: Depth Anything in
Any Direction, an accurate, zero-shot generalizable, and
fully end-to-end panoramic depth estimator. Specifically, for scaling up
panoramic data, we introduce a data curation engine for generating high-quality
panoramic depth data from perspective, and create sim543K panoramic
RGB-depth pairs, bringing the total to sim607K. To further mitigate the
spherical distortions, we present SphereViT, which explicitly leverages
spherical coordinates to enforce the spherical geometric consistency in
panoramic image features, yielding improved performance. A comprehensive
benchmark on multiple datasets clearly demonstrates DA^{2}'s SoTA
performance, with an average 38% improvement on AbsRel over the strongest
zero-shot baseline. Surprisingly, DA^{2} even outperforms prior in-domain
methods, highlighting its superior zero-shot generalization. Moreover, as an
end-to-end solution, DA^{2} exhibits much higher efficiency over fusion-based
approaches. Both the code and the curated panoramic data will be released.
Project page: https://depth-any-in-any-dir.github.io/.
68dcbd594159d1f2418f9c0e
Haodong Li
claimed_verified
2025-10-01T10:20:53.957Z
641d211e353524fe41f16387
/avatars/c6e72c82c029b415a035beebee50b52c.svg
Haodong Li
user
haodongli
68dcbd594159d1f2418f9c0f
Wangguangdong Zheng
68dcbd594159d1f2418f9c10
Jing He
68dcbd594159d1f2418f9c11
Yuhao Liu
68dcbd594159d1f2418f9c12
Xin Lin
68dcbd594159d1f2418f9c13
Xin Yang
68dcbd594159d1f2418f9c14
Ying-Cong Chen
68dcbd594159d1f2418f9c15
Chunchao Guo
DA^2: Depth Anything in Any Direction
Panorama has a full FoV (360^circtimes180^circ), offering a more
complete visual description than perspective images. Thanks to this
characteristic, panoramic depth estimation is gaining increasing traction in 3D
vision. However, due to the scarcity of panoramic data, previous methods are
often restricted to in-domain settings, leading to poor zero-shot
generalization. Furthermore, due to the spherical distortions inherent in
panoramas, many approaches rely on perspective splitting (e.g., cubemaps),
which leads to suboptimal efficiency. To address these challenges, we propose
DA^{2}: Depth Anything in
Any Direction, an accurate, zero-shot generalizable, and
fully end-to-end panoramic depth estimator. Specifically, for scaling up
panoramic data, we introduce a data curation engine for generating high-quality
panoramic depth data from perspective, and create sim543K panoramic
RGB-depth pairs, bringing the total to sim607K. To further mitigate the
spherical distortions, we present SphereViT, which explicitly leverages
spherical coordinates to enforce the spherical geometric consistency in
panoramic image features, yielding improved performance. A comprehensive
benchmark on multiple datasets clearly demonstrates DA^{2}'s SoTA
performance, with an average 38% improvement on AbsRel over the strongest
zero-shot baseline. Surprisingly, DA^{2} even outperforms prior in-domain
methods, highlighting its superior zero-shot generalization. Moreover, as an
end-to-end solution, DA^{2} exhibits much higher efficiency over fusion-based
approaches. Both the code and the curated panoramic data will be released.
Project page: https://depth-any-in-any-dir.github.io/.
DA², a zero-shot generalizable and fully end-to-end panoramic depth estimator, addresses challenges in panoramic depth estimation by using a data curation engine and SphereViT to handle spherical distortions, achieving state-of-the-art performance.
panoramic depth estimation 3D vision zero-shot generalization perspective splitting cubemaps data curation engine SphereViT spherical coordinates spherical geometric consistency AbsRel in-domain methods fusion-based approaches