Learning to Learn with Contrastive Meta-Objective
Shiguang Wu, Yaqing Wang, Yatao Bian, Quanming Yao
Meta-learning enables learning systems to adapt quickly to new tasks, similar to humans.Different meta-learning approaches all work under/with the mini-batch episodic training framework. Such framework naturally gives the information about task identity, which can serve as additional supervision for meta-training to improve generalizability. We propose to exploit task identity as additional supervision in meta-training, inspired by the alignment and discrimination ability which is is intrinsic in human's fast learning.This is achieved by contrasting what meta-learners learn, i.e., model representations.The proposed ConML is evaluating and optimizing the contrastive meta-objective under a problem- and learner-agnostic meta-training framework.We demonstrate that ConML integrates seamlessly with existing meta-learners, as well as in-context learning models, and brings significant boost in performance with small implementation cost.Code is provided at https://anonymous.4open.science/r/conml_ano-3372.