AdvEvo-MARL: Shaping Internalized Safety through Adversarial
  Co-Evolution in Multi-Agent Reinforcement Learning
LLM-based multi-agent systems excel at planning, tool use, and role
coordination, but their openness and interaction complexity also expose them to
jailbreak, prompt-injection, and adversarial collaboration. Existing defenses
fall into two lines: (i) self-verification that asks each agent to pre-filter
unsafe instructions before execution, and (ii) external guard modules that
police behaviors. The former often underperforms because a standalone agent
lacks sufficient capacity to detect cross-agent unsafe chains and
delegation-induced risks; the latter increases system overhead and creates a
single-point-of-failure-once compromised, system-wide safety collapses, and
adding more guards worsens cost and complexity. To solve these challenges, we
propose AdvEvo-MARL, a co-evolutionary multi-agent reinforcement learning
framework that internalizes safety into task agents. Rather than relying on
external guards, AdvEvo-MARL jointly optimizes attackers (which synthesize
evolving jailbreak prompts) and defenders (task agents trained to both
accomplish their duties and resist attacks) in adversarial learning
environments. To stabilize learning and foster cooperation, we introduce a
public baseline for advantage estimation: agents within the same functional
group share a group-level mean-return baseline, enabling lower-variance updates
and stronger intra-group coordination. Across representative attack scenarios,
AdvEvo-MARL consistently keeps attack-success rate (ASR) below 20%, whereas
baselines reach up to 38.33%, while preserving-and sometimes improving-task
accuracy (up to +3.67% on reasoning tasks). These results show that safety and
utility can be jointly improved without relying on extra guard agents or added
system overhead.
68e49051e4e093a7044e4d9d
Zhenyu Pan
68e49051e4e093a7044e4d9e
Yiting Zhang
68e49051e4e093a7044e4d9f
Zhuo Liu
68e49051e4e093a7044e4da0
Yolo Yunlong Tang
68e49051e4e093a7044e4da1
Zeliang Zhang
68e49051e4e093a7044e4da2
Haozheng Luo
68e49051e4e093a7044e4da3
Yuwei Han
68e49051e4e093a7044e4da4
Jianshu Zhang
68e49051e4e093a7044e4da5
Dennis Wu
68e49051e4e093a7044e4da6
Hong-Yu Chen
68e49051e4e093a7044e4da7
Haoran Lu
68e49051e4e093a7044e4da8
Haoyang Fang
68e49051e4e093a7044e4da9
Manling Li
68e49051e4e093a7044e4daa
Chenliang Xu
68e49051e4e093a7044e4dab
Philip S. Yu
68e49051e4e093a7044e4dac
Han Liu
AdvEvo-MARL: Shaping Internalized Safety through Adversarial
  Co-Evolution in Multi-Agent Reinforcement Learning
LLM-based multi-agent systems excel at planning, tool use, and role
coordination, but their openness and interaction complexity also expose them to
jailbreak, prompt-injection, and adversarial collaboration. Existing defenses
fall into two lines: (i) self-verification that asks each agent to pre-filter
unsafe instructions before execution, and (ii) external guard modules that
police behaviors. The former often underperforms because a standalone agent
lacks sufficient capacity to detect cross-agent unsafe chains and
delegation-induced risks; the latter increases system overhead and creates a
single-point-of-failure-once compromised, system-wide safety collapses, and
adding more guards worsens cost and complexity. To solve these challenges, we
propose AdvEvo-MARL, a co-evolutionary multi-agent reinforcement learning
framework that internalizes safety into task agents. Rather than relying on
external guards, AdvEvo-MARL jointly optimizes attackers (which synthesize
evolving jailbreak prompts) and defenders (task agents trained to both
accomplish their duties and resist attacks) in adversarial learning
environments. To stabilize learning and foster cooperation, we introduce a
public baseline for advantage estimation: agents within the same functional
group share a group-level mean-return baseline, enabling lower-variance updates
and stronger intra-group coordination. Across representative attack scenarios,
AdvEvo-MARL consistently keeps attack-success rate (ASR) below 20%, whereas
baselines reach up to 38.33%, while preserving-and sometimes improving-task
accuracy (up to +3.67% on reasoning tasks). These results show that safety and
utility can be jointly improved without relying on extra guard agents or added
system overhead.
AdvEvo-MARL, a co-evolutionary multi-agent reinforcement learning framework, enhances safety and utility in LLM-based multi-agent systems by internally optimizing task agents against evolving attacks without additional overhead.
multi-agent systems planning tool use role coordination jailbreak prompt-injection adversarial collaboration self-verification external guard modules co-evolutionary reinforcement learning attackers defenders adversarial learning advantage estimation group-level mean-return baseline attack-success rate task accuracy reasoning tasks