CoMAS: Co-Evolving Multi-Agent Systems via Interaction Rewards
Self-evolution is a central research topic in enabling large language model
(LLM)-based agents to continually improve their capabilities after pretraining.
Recent research has witnessed a transition from reinforcement learning
(RL)-free to RL-based methods. Current RL-based methods either rely on dense
external reward signals or extract intrinsic reward signals from LLMs
themselves. However, these approaches diverge from the self-evolution
mechanisms observed in human intelligence, where individuals learn and improve
through mutual discussion and collaboration. In this work, we introduce
Co-Evolving Multi-Agent Systems (CoMAS), a novel framework that enables agents
to improve autonomously by learning from inter-agent interactions without
external supervision. CoMAS generates intrinsic rewards from rich discussion
dynamics, employs an LLM-as-a-judge mechanism to formulate these rewards, and
optimizes each agent's policy through RL, thereby enabling decentralized and
scalable co-evolution. Experimental results demonstrate that CoMAS consistently
outperforms untrained agents and achieves state-of-the-art performance across
most evaluation settings. Ablation studies confirm the necessity of
interaction-based reward signals and reveal promising scalability as the number
and diversity of agents increase. These findings establish CoMAS as a novel and
effective paradigm for self-evolution in LLM-based agents.
68e891e295e8e6771df38b41
Xiangyuan Xue
claimed_verified
2025-10-10T08:54:27.924Z
65082baabc8788c4064d5360
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/NZpFVnTpPGcCe8mvFMD-L.jpeg
Xiangyuan Xue
user
xxyQwQ
68e891e295e8e6771df38b42
Yifan Zhou
68e891e295e8e6771df38b43
Guibin Zhang
68e891e295e8e6771df38b44
Zaibin Zhang
68e891e295e8e6771df38b45
Yijiang Li
68e891e295e8e6771df38b46
Chen Zhang
68e891e295e8e6771df38b47
Zhenfei Yin
68e891e295e8e6771df38b48
Philip Torr
68e891e295e8e6771df38b49
Wanli Ouyang
68e891e295e8e6771df38b4a
Lei Bai
CoMAS: Co-Evolving Multi-Agent Systems via Interaction Rewards
Self-evolution is a central research topic in enabling large language model
(LLM)-based agents to continually improve their capabilities after pretraining.
Recent research has witnessed a transition from reinforcement learning
(RL)-free to RL-based methods. Current RL-based methods either rely on dense
external reward signals or extract intrinsic reward signals from LLMs
themselves. However, these approaches diverge from the self-evolution
mechanisms observed in human intelligence, where individuals learn and improve
through mutual discussion and collaboration. In this work, we introduce
Co-Evolving Multi-Agent Systems (CoMAS), a novel framework that enables agents
to improve autonomously by learning from inter-agent interactions without
external supervision. CoMAS generates intrinsic rewards from rich discussion
dynamics, employs an LLM-as-a-judge mechanism to formulate these rewards, and
optimizes each agent's policy through RL, thereby enabling decentralized and
scalable co-evolution. Experimental results demonstrate that CoMAS consistently
outperforms untrained agents and achieves state-of-the-art performance across
most evaluation settings. Ablation studies confirm the necessity of
interaction-based reward signals and reveal promising scalability as the number
and diversity of agents increase. These findings establish CoMAS as a novel and
effective paradigm for self-evolution in LLM-based agents.
Co-Evolving Multi-Agent Systems (CoMAS) enable LLM-based agents to improve autonomously through inter-agent interactions and intrinsic rewards, achieving state-of-the-art performance.
self-evolution large language model (LLM) reinforcement learning (RL) intrinsic reward signals Co-Evolving Multi-Agent Systems (CoMAS) LLM-as-a-judge decentralized scalable co-evolution