DOVTrack: Data-Efficient Open-Vocabulary Tracking
Zekun Qian, Ruize Han, Zhixiang Wang, Junhui Hou, Wei Feng
Open-Vocabulary Multi-Object Tracking (OVMOT) aims to detect and track multi-category objects that have both been seen and not seen during training.Currently, a significant challenge in this domain is the lack of large-scale annotated video data for training.To address this challenge, this work aims to effectively train the OV tracker using only the existing limited and sparse-annotated video data.Specifically, on the one hand, for the association task, we apply the diffusion models to complement the missing objects at the gaps between annotated frames, which is achieved directly in the implicit embedding space and enhances the coherence and accuracy of association features.On the other hand, for the object detection task, we develop a dynamic group contrastive learning strategy for object recognition and a simple yet effective loss, namely adaptive localization loss.Extensive results verify that the proposed method achieves promising results on the OVMOT benchmark, without additional data or annotations.