Depth Anywhere: Enhancing 360 Monocular Depth Estimation via Perspective
  Distillation and Unlabeled Data Augmentation
Accurately estimating depth in 360-degree imagery is crucial for virtual
reality, autonomous navigation, and immersive media applications. Existing
depth estimation methods designed for perspective-view imagery fail when
applied to 360-degree images due to different camera projections and
distortions, whereas 360-degree methods perform inferior due to the lack of
labeled data pairs. We propose a new depth estimation framework that utilizes
unlabeled 360-degree data effectively. Our approach uses state-of-the-art
perspective depth estimation models as teacher models to generate pseudo labels
through a six-face cube projection technique, enabling efficient labeling of
depth in 360-degree images. This method leverages the increasing availability
of large datasets. Our approach includes two main stages: offline mask
generation for invalid regions and an online semi-supervised joint training
regime. We tested our approach on benchmark datasets such as Matterport3D and
Stanford2D3D, showing significant improvements in depth estimation accuracy,
particularly in zero-shot scenarios. Our proposed training pipeline can enhance
any 360 monocular depth estimator and demonstrates effective knowledge transfer
across different camera projections and data types. See our project page for
results: https://albert100121.github.io/Depth-Anywhere/
6672e9972376de4b7ab47afd
Ning-Hsu Wang
extracted_confirmed
2024-06-19T14:36:18.321Z
64647db4fdbe694bc7537504
/avatars/4df13f878599bc7310ddecdb29a381b0.svg
Albert Wang
user
Albert-NHWang
6672e9972376de4b7ab47afe
Yu-Lun Liu
Depth Anywhere: Enhancing 360 Monocular Depth Estimation via Perspective
  Distillation and Unlabeled Data Augmentation
Accurately estimating depth in 360-degree imagery is crucial for virtual
reality, autonomous navigation, and immersive media applications. Existing
depth estimation methods designed for perspective-view imagery fail when
applied to 360-degree images due to different camera projections and
distortions, whereas 360-degree methods perform inferior due to the lack of
labeled data pairs. We propose a new depth estimation framework that utilizes
unlabeled 360-degree data effectively. Our approach uses state-of-the-art
perspective depth estimation models as teacher models to generate pseudo labels
through a six-face cube projection technique, enabling efficient labeling of
depth in 360-degree images. This method leverages the increasing availability
of large datasets. Our approach includes two main stages: offline mask
generation for invalid regions and an online semi-supervised joint training
regime. We tested our approach on benchmark datasets such as Matterport3D and
Stanford2D3D, showing significant improvements in depth estimation accuracy,
particularly in zero-shot scenarios. Our proposed training pipeline can enhance
any 360 monocular depth estimator and demonstrates effective knowledge transfer
across different camera projections and data types. See our project page for
results: https://albert100121.github.io/Depth-Anywhere/
A new depth estimation framework for 360-degree imagery uses pseudo-labels generated by perspective depth models to improve accuracy, especially in zero-shot scenarios.
depth estimation 360-degree imagery perspective-view imagery camera projections distortions pseudo labels six-face cube projection mask generation semi-supervised joint training Matterport3D Stanford2D3D monocular depth estimator knowledge transfer