Heterogeneous Adversarial Play in Interactive Environments
Manjie Xu, Xinyi Yang, Jiayu Zhan, Wei Liang, Chi Zhang, Yixin Zhu
Self-play acts as a central role in enabling the autonomous acquisition of complex knowledge in structured environments, particularly for challenging tasks with compositional spaces where well-annotated data is limited. However, in more general learning scenarios, such symmetry for bootstrapping is often absent. For instance, in human education, teachers serve dual roles: as knowledge providers and problem proposers. They gradually present students with increasingly difficult problems as the students progress in mastering complex concepts. Although the roles of teacher and student are inherently different, they can be viewed as an adversarial pair, where the teacher challenges the student with problems, and the student responds by solving them. Inspired by this adversarial dynamic, we propose Heterogeneous Adversarial Play (HAP), an adversarial learning framework in which a teacher module generates problems tailored to the student's current capabilities. The two agents engage in a zero-sum game, driving the student's learning progress. Through multi-task learning experiments, we demonstrate that our framework achieves competitive performance relative to state-of-the-art baselines and facilitates more effective curriculum design for human learners. Our results highlight the potential of HAP to enhance the efficiency and robustness of AI learning systems.