UFM: A Simple Path towards Unified Dense Correspondence with Flow
Dense image correspondence is central to many applications, such as visual
odometry, 3D reconstruction, object association, and re-identification.
Historically, dense correspondence has been tackled separately for
wide-baseline scenarios and optical flow estimation, despite the common goal of
matching content between two images. In this paper, we develop a Unified Flow &
Matching model (UFM), which is trained on unified data for pixels that are
co-visible in both source and target images. UFM uses a simple, generic
transformer architecture that directly regresses the (u,v) flow. It is easier
to train and more accurate for large flows compared to the typical
coarse-to-fine cost volumes in prior work. UFM is 28% more accurate than
state-of-the-art flow methods (Unimatch), while also having 62% less error and
6.7x faster than dense wide-baseline matchers (RoMa). UFM is the first to
demonstrate that unified training can outperform specialized approaches across
both domains. This result enables fast, general-purpose correspondence and
opens new directions for multi-modal, long-range, and real-time correspondence
tasks.
684ae1efdbd21a9cc27b0f19
Yuchen Zhang
684ae1efdbd21a9cc27b0f1a
Nikhil Keetha
684ae1efdbd21a9cc27b0f1b
Chenwei Lyu
684ae1efdbd21a9cc27b0f1c
Bhuvan Jhamb
684ae1efdbd21a9cc27b0f1d
Yutian Chen
684ae1efdbd21a9cc27b0f1e
Yuheng Qiu
684ae1efdbd21a9cc27b0f1f
Jay Karhade
684ae1efdbd21a9cc27b0f20
Shreyas Jha
684ae1efdbd21a9cc27b0f21
Yaoyu Hu
684ae1efdbd21a9cc27b0f22
Deva Ramanan
684ae1efdbd21a9cc27b0f23
Sebastian Scherer
684ae1efdbd21a9cc27b0f24
Wenshan Wang
UFM: A Simple Path towards Unified Dense Correspondence with Flow
Dense image correspondence is central to many applications, such as visual
odometry, 3D reconstruction, object association, and re-identification.
Historically, dense correspondence has been tackled separately for
wide-baseline scenarios and optical flow estimation, despite the common goal of
matching content between two images. In this paper, we develop a Unified Flow &
Matching model (UFM), which is trained on unified data for pixels that are
co-visible in both source and target images. UFM uses a simple, generic
transformer architecture that directly regresses the (u,v) flow. It is easier
to train and more accurate for large flows compared to the typical
coarse-to-fine cost volumes in prior work. UFM is 28% more accurate than
state-of-the-art flow methods (Unimatch), while also having 62% less error and
6.7x faster than dense wide-baseline matchers (RoMa). UFM is the first to
demonstrate that unified training can outperform specialized approaches across
both domains. This result enables fast, general-purpose correspondence and
opens new directions for multi-modal, long-range, and real-time correspondence
tasks.
A Unified Flow & Matching model (UFM) improves dense image correspondence accuracy and speed by using a transformer architecture for unified data training, outperforming specialized methods for both optical flow and wide-baseline scenarios.
transformer architecture dense image correspondence visual odometry 3D reconstruction object association re-identification wide-baseline scenarios optical flow estimation co-visible pixels (u v) flow coarse-to-fine cost volumes Unimatch RoMa