Dual Alignment Framework for Few-shot Learning with Inter-Set and Intra-Set Shifts
Siyang Jiang, Rui Fang, Hsi-Wen Chen, Wei Ding, Guoliang Xing, Ming-syan Chen
Few-shot learning (FSL) aims to classify unseen examples (query set) into labeled data (support set) through low-dimensional embeddings. However, the diversity and unpredictability of environments and capture devices make FSL more challenging in real-world applications. In this paper, we propose $\textit{Dual Support Query Shift (DSQS)}$, a novel challenge in FSL that integrates two key issues: inter-set shifts (between support and query sets) and intra-set shifts (within each set), which significantly hinder model performance. To tackle these challenges, we introduce a  $\textit{DUal ALignment framework (DUAL)}$, which core insight is that clean features can improve optimal transportation (OT) alignment. Firstly, DUAL leverages a robust embedding function enhanced by a repairer network trained with perturbed and adversarially generated "hard" examples to obtain clean features. Additionally, it incorporates a two-stage OT approach with a negative entropy regularizer, aligning support set instances, minimizing intra-class distances, and using query data as anchor nodes for achieving effective distribution alignment. We provide a theoretical bound of DUAL and experimental results on three image datasets, compared against 10 state-of-the-art baselines, showing that DUAL achieves a remarkable average performance improvement of $18.39\%$. Our code is available at https://anonymous.4open.science/r/CODE-F558/.