Depth Pro: Sharp Monocular Metric Depth in Less Than a Second
We present a foundation model for zero-shot metric monocular depth
estimation. Our model, Depth Pro, synthesizes high-resolution depth maps with
unparalleled sharpness and high-frequency details. The predictions are metric,
with absolute scale, without relying on the availability of metadata such as
camera intrinsics. And the model is fast, producing a 2.25-megapixel depth map
in 0.3 seconds on a standard GPU. These characteristics are enabled by a number
of technical contributions, including an efficient multi-scale vision
transformer for dense prediction, a training protocol that combines real and
synthetic datasets to achieve high metric accuracy alongside fine boundary
tracing, dedicated evaluation metrics for boundary accuracy in estimated depth
maps, and state-of-the-art focal length estimation from a single image.
Extensive experiments analyze specific design choices and demonstrate that
Depth Pro outperforms prior work along multiple dimensions. We release code and
weights at https://github.com/apple/ml-depth-pro
66ff615b81382f012c08a66f
Aleksei Bochkovskii
claimed_verified
2024-10-04T20:11:27.181Z
66fffa681c6cda57caf8b0b9
/avatars/61ebf62562ce4432667b8be2f713f707.svg
Aleksei Bochkovskii
user
alexeyab84
66ff615b81382f012c08a670
Amaël Delaunoy
claimed_verified
2024-10-04T04:41:38.750Z
665ebfb7a57b277d7a202fb1
/avatars/15538783a4fbc4688db85d1b49cd24e2.svg
Amael Delaunoy
user
amael-apple
66ff615b81382f012c08a671
Hugo Germain
66ff615b81382f012c08a672
Marcel Santos
admin_assigned
2024-10-04T08:46:54.308Z
65986fd3f0102bce68e15717
/avatars/da161214fb43d8bc3c07f98df0554e73.svg
Marcel Santoso
user
msantoso98
66ff615b81382f012c08a673
Yichao Zhou
66ff615b81382f012c08a674
Stephan R. Richter
claimed_verified
2024-10-04T07:23:22.709Z
66ff713ea98cf0a69f8d5cd0
/avatars/4b77834fce727f3af99f8fb77e646609.svg
Stephan Richter
user
srrichter
66ff615b81382f012c08a675
Vladlen Koltun
claimed_verified
2024-10-10T10:59:52.185Z
6707b1a9356f027ca986d6b1
/avatars/d32eae81b27dc4f7090196d410e96876.svg
Vladlen Koltun
user
vkoltun
Depth Pro: Sharp Monocular Metric Depth in Less Than a Second
We present a foundation model for zero-shot metric monocular depth
estimation. Our model, Depth Pro, synthesizes high-resolution depth maps with
unparalleled sharpness and high-frequency details. The predictions are metric,
with absolute scale, without relying on the availability of metadata such as
camera intrinsics. And the model is fast, producing a 2.25-megapixel depth map
in 0.3 seconds on a standard GPU. These characteristics are enabled by a number
of technical contributions, including an efficient multi-scale vision
transformer for dense prediction, a training protocol that combines real and
synthetic datasets to achieve high metric accuracy alongside fine boundary
tracing, dedicated evaluation metrics for boundary accuracy in estimated depth
maps, and state-of-the-art focal length estimation from a single image.
Extensive experiments analyze specific design choices and demonstrate that
Depth Pro outperforms prior work along multiple dimensions. We release code and
weights at https://github.com/apple/ml-depth-pro
Depth Pro is a fast and accurate zero-shot monocular depth estimation model that generates high-resolution depth maps using a multi-scale transformer and a combined real-synthetic training protocol.
zero-shot monocular depth estimation high-resolution depth maps multi-scale vision transformer dense prediction real and synthetic datasets boundary accuracy focal length estimation