COGNAC: Cooperative Graph-based Networked Agent Challenges for Multi-Agent Reinforcement Learning
Jules Sintes, Ana Busic
Many controlled complex systems have an inherent network structure such as power grids, traffic lights systems or computer networks. Automatically controlling these systems is highly challenging due to their combinatorial complexity. Standard single-agent reinforcement learning (RL) approaches often struggle with the curse of dimensionality in such settings. In contrast, the multi-agent paradigm offers a promising solution by distributing decision-making, thereby addressing both algorithmic and combinatorial challenges. In this paper, we introduce COGNAC (COoperative Graph-based Networked Agent Challenges), the first collection of benchmark environments specifically designed for fully cooperative multi-agent tasks with graph structures. COGNAC bridges the gap between theoretical research in network control and practical multi-agent RL applications by offering a flexible, scalable platform with a suite of simple yet highly challenging problems rooted in networked environments. Our benchmarks also support the development and evaluation of decentralized and distributed learning algorithms, motivated by the growing interest in more sustainable and frugal AI systems. We evaluate standard independent learning algorithms on COGNAC environments and show that they outperform both heuristic-based baselines and centralized approaches across varying problem sizes.