FlowFeat: Pixel-Dense Embedding of Motion Profiles
Nikita Araslanov, Anna Sonnweber, Daniel Cremers
Dense and versatile image representations underpin the success of virtually any computer vision application. However, state-of-the-art networks, such as transformers, produce low-resolution feature grids, which are suboptimal for dense prediction tasks. Addressing this limitation, our work presents FlowFeat, a high-resolution and multi-task feature representation. The key ingredient behind FlowFeat is a novel distillation technique embedding a distribution of plausible apparent motion, or *motion profiles*. Leveraging optical flow networks and diverse video data, we develop an effective self-supervised training framework, which statistically approximates the apparent motion. With a remarkable level of spatial detail, FlowFeat encodes a compelling degree of geometric and semantic cues while exhibiting high temporal consistency. Empirically, FlowFeat significantly enhances the representation power of five state-of-the-art encoders and alternative upsampling strategies across three dense tasks: video object segmentation, monocular depth estimation and semantic segmentation.Training FlowFeat is robust to inaccurate flow estimation and remains highly effective also with unsupervised flow networks.Our work is a confident step forward towards reliable dense and versatile image representations.