Depth Pro: Sharp Monocular Metric Depth in Less Than a Second
We present a foundation model for zero-shot metric monocular depth
estimation. Our model, Depth Pro, synthesizes high-resolution depth maps with
unparalleled sharpness and high-frequency details. The predictions are metric,
with absolute scale, without relying on the availability of metadata such as
camera intrinsics. And the model is fast, producing a 2.25-megapixel depth map
in 0.3 seconds on a standard GPU. These characteristics are enabled by a number
of technical contributions, including an efficient multi-scale vision
transformer for dense prediction, a training protocol that combines real and
synthetic datasets to achieve high metric accuracy alongside fine boundary
tracing, dedicated evaluation metrics for boundary accuracy in estimated depth
maps, and state-of-the-art focal length estimation from a single image.
Extensive experiments analyze specific design choices and demonstrate that
Depth Pro outperforms prior work along multiple dimensions. We release code and
weights at https://github.com/apple/ml-depth-pro
66ff615b81382f012c08a66f
Aleksei Bochkovskii
claimed_verified
2024-10-04T20:11:27.181Z
66fffa681c6cda57caf8b0b9
/avatars/61ebf62562ce4432667b8be2f713f707.svg
Aleksei Bochkovskii
user
alexeyab84
66ff615b81382f012c08a670
Amaël Delaunoy
claimed_verified
2024-10-04T04:41:38.750Z
665ebfb7a57b277d7a202fb1
/avatars/15538783a4fbc4688db85d1b49cd24e2.svg
Amael Delaunoy
user
amael-apple
66ff615b81382f012c08a671
Hugo Germain
66ff615b81382f012c08a672
Marcel Santos
admin_assigned
2024-10-04T08:46:54.308Z
65986fd3f0102bce68e15717
/avatars/da161214fb43d8bc3c07f98df0554e73.svg
Marcel Santoso
user
msantoso98
66ff615b81382f012c08a673
Yichao Zhou
66ff615b81382f012c08a674
Stephan R. Richter
claimed_verified
2024-10-04T07:23:22.709Z
66ff713ea98cf0a69f8d5cd0
/avatars/4b77834fce727f3af99f8fb77e646609.svg
Stephan Richter
user
srrichter
66ff615b81382f012c08a675
Vladlen Koltun
claimed_verified
2024-10-10T10:59:52.185Z
6707b1a9356f027ca986d6b1
/avatars/d32eae81b27dc4f7090196d410e96876.svg
Vladlen Koltun
user
vkoltun
Depth Pro: Sharp Monocular Metric Depth in Less Than a Second
We present a foundation model for zero-shot metric monocular depth
estimation. Our model, Depth Pro, synthesizes high-resolution depth maps with
unparalleled sharpness and high-frequency details. The predictions are metric,
with absolute scale, without relying on the availability of metadata such as
camera intrinsics. And the model is fast, producing a 2.25-megapixel depth map
in 0.3 seconds on a standard GPU. These characteristics are enabled by a number
of technical contributions, including an efficient multi-scale vision
transformer for dense prediction, a training protocol that combines real and
synthetic datasets to achieve high metric accuracy alongside fine boundary
tracing, dedicated evaluation metrics for boundary accuracy in estimated depth
maps, and state-of-the-art focal length estimation from a single image.
Extensive experiments analyze specific design choices and demonstrate that
Depth Pro outperforms prior work along multiple dimensions. We release code and
weights at https://github.com/apple/ml-depth-pro
zero-shot metric monocular depth estimation high-resolution depth maps absolute scale camera intrinsics efficient multi-scale vision transformer dense prediction training protocol real and synthetic datasets high metric accuracy fine boundary tracing dedicated evaluation metrics for boundary accuracy depth maps focal length estimation single image state-of-the-art