Wonder Wins Ways: Curiosity-Driven Exploration through Multi-Agent Contextual Calibration
Yiyuan Pan, Zhe Liu, Hesheng Wang
Unlocking autonomous exploration in complex Multi-Agent Reinforcement Learning (MARL) systems, especially under sparse rewards, hinges on endowing agents with effective motivation. While artificial curiosity offers a powerful self-supervised drive, its efficacy is undermined when agents cannot robustly discern meaningful novelty from environmental stochasticity. Existing methods overlook the crucial need for agents to dynamically calibrate their intrinsic curiosity based on a learned understanding of their peers, leading to suboptimal performance in truly decentralized and communication-less settings. Inspired by how human children rapidly master new social games by observing peers and adaptively calibrating their own exploratory behaviors, we propose a novel approach to enhance multi-agent exploration. We introduce CERMIC, a principled framework that empowers agents to robustly filter noisy surprise signals and guide exploration by dynamically calibrating their intrinsic curiosity with inferred multi-agent contextual awareness. Additionally, CERMIC generates theoretically-grounded intrinsic rewards, encouraging agents to explore state transitions with high information gain. Finally, we evaluate CERMIC on benchmark suites including VMAS, MeltingPot, and SMACv2. Empirical results demonstrate that exploration with CERMIC significantly outperforms SoTA algorithms in sparse-reward environments.