DepthFM: Fast Monocular Depth Estimation with Flow Matching
Monocular depth estimation is crucial for numerous downstream vision tasks
and applications. Current discriminative approaches to this problem are limited
due to blurry artifacts, while state-of-the-art generative methods suffer from
slow sampling due to their SDE nature. Rather than starting from noise, we seek
a direct mapping from input image to depth map. We observe that this can be
effectively framed using flow matching, since its straight trajectories through
solution space offer efficiency and high quality. Our study demonstrates that a
pre-trained image diffusion model can serve as an adequate prior for a flow
matching depth model, allowing efficient training on only synthetic data to
generalize to real images. We find that an auxiliary surface normals loss
further improves the depth estimates. Due to the generative nature of our
approach, our model reliably predicts the confidence of its depth estimates. On
standard benchmarks of complex natural scenes, our lightweight approach
exhibits state-of-the-art performance at favorable low computational cost
despite only being trained on little synthetic data.
65fbbae8030389a29b7b9c68
Ming Gui
claimed_verified
2024-09-30T08:10:02.718Z
6319f120d47392f426cd48be
/avatars/fa6d8761ef7a1a86a8a7e97408dc3142.svg
Ming Gui
user
mgui
65fbbae8030389a29b7b9c69
Johannes S. Fischer
claimed_verified
2024-03-22T09:30:16.872Z
63cea5dd257cc48a04f8dc21
/avatars/db697e0954ff5be392b8ab588df2f0c4.svg
J. Schusterbauer
user
joh-schb
65fbbae8030389a29b7b9c6a
Ulrich Prestel
admin_assigned
2024-03-21T10:22:30.697Z
6425e56785f26ab94af19797
/avatars/555fafac9b555da7be1a112fac9a0cbf.svg
Ulrich Prestel
user
upr
65fbbae8030389a29b7b9c6b
Pingchuan Ma
65fbbae8030389a29b7b9c6c
Dmytro Kotovenko
65fbbae8030389a29b7b9c6d
Olga Grebenkova
claimed_verified
2024-07-08T20:01:18.911Z
633b481d0d68f86e2d96ed84
https://cdn-avatars.huggingface.co/v1/production/uploads/633b481d0d68f86e2d96ed84/ACvGaMbeYSYIGiez0jJrE.jpeg
Olga Grebenkova
user
magistrkoljan
65fbbae8030389a29b7b9c6e
Stefan Andreas Baumann
claimed_verified
2024-03-25T09:44:14.282Z
6471c12a0c2b5fdaf1f07c45
/avatars/6783067212d24d1e716a8b4c64df61b4.svg
Stefan Baumann
user
stefan-baumann
65fbbae8030389a29b7b9c6f
Vincent Tao Hu
admin_assigned
2024-03-21T10:24:39.243Z
63dcd7ac22cc06e76a8484ce
https://cdn-avatars.huggingface.co/v1/production/uploads/63dcd7ac22cc06e76a8484ce/4O8NcevQ0ZWhJJ1r4Pbam.jpeg
Vincent Tao Hu
user
taohu
65fbbae8030389a29b7b9c70
Björn Ommer
DepthFM: Fast Monocular Depth Estimation with Flow Matching
Monocular depth estimation is crucial for numerous downstream vision tasks
and applications. Current discriminative approaches to this problem are limited
due to blurry artifacts, while state-of-the-art generative methods suffer from
slow sampling due to their SDE nature. Rather than starting from noise, we seek
a direct mapping from input image to depth map. We observe that this can be
effectively framed using flow matching, since its straight trajectories through
solution space offer efficiency and high quality. Our study demonstrates that a
pre-trained image diffusion model can serve as an adequate prior for a flow
matching depth model, allowing efficient training on only synthetic data to
generalize to real images. We find that an auxiliary surface normals loss
further improves the depth estimates. Due to the generative nature of our
approach, our model reliably predicts the confidence of its depth estimates. On
standard benchmarks of complex natural scenes, our lightweight approach
exhibits state-of-the-art performance at favorable low computational cost
despite only being trained on little synthetic data.
A flow matching approach using pre-trained image diffusion models achieves state-of-the-art monocular depth estimation with high quality and low computational cost, leveraging synthetic data and surface normals loss.
monocular depth estimation discriminative approaches generative methods stochastic differential equations (SDE) flow matching pre-trained image diffusion model straight trajectories solution space auxiliary surface normals loss