DON’T NEED RETRAINING: A Mixture of DETR and Vision Foundation Models for Cross-Domain Few-Shot Object Detection
Changhan Liu, xunzhi xiang, Zixuan Duan, Wenbin Li, Yang Gao, Qi Fan
Cross-Domain Few-Shot Object Detection (CD-FSOD) seeks to generalize to unseen domains using only a limited number of annotated samples from the target domain. This requires models to possess both strong generalization and regression capabilities. However, existing well-trained detection models and vision foundation models (VFMs) are typically limited, excelling in either localization or generalization, but rarely both, making them unsuitable for the CD-FSOD task. In this paper, we propose a novel CD-FSOD paradigm that retains the structure and localization capabilities of well-trained object detection models while achieving the transfer of generalization abilities from VFMs with minimal additional parameters. Specifically, our proposed Mixture-of-Experts (MoE) based method dynamically routes and modulates VFM features as representation experts to facilitate traditional object detection models for CD-FSOD. Furthermore, our approach eliminates the traditional two-stage paradigm of base training followed by novel fine-tuning. Instead, it extends well-trained object detection models to detect novel classes in unseen domains without requiring re-training on the base data. Experimental results on multiple cross-domain datasets validate the effectiveness of our method.