Tree-based Dialogue Reinforced Policy Optimization for Red-Teaming
  Attacks
Despite recent rapid progress in AI safety, current large language models
remain vulnerable to adversarial attacks in multi-turn interaction settings,
where attackers strategically adapt their prompts across conversation turns and
pose a more critical yet realistic challenge. Existing approaches that discover
safety vulnerabilities either rely on manual red-teaming with human experts or
employ automated methods using pre-defined templates and human-curated attack
data, with most focusing on single-turn attacks. However, these methods did not
explore the vast space of possible multi-turn attacks, failing to consider
novel attack trajectories that emerge from complex dialogue dynamics and
strategic conversation planning. This gap is particularly critical given recent
findings that LLMs exhibit significantly higher vulnerability to multi-turn
attacks compared to single-turn attacks. We propose DialTree-RPO, an on-policy
reinforcement learning framework integrated with tree search that autonomously
discovers diverse multi-turn attack strategies by treating the dialogue as a
sequential decision-making problem, enabling systematic exploration without
manually curated data. Through extensive experiments, our approach not only
achieves more than 25.9% higher ASR across 10 target models compared to
previous state-of-the-art approaches, but also effectively uncovers new attack
strategies by learning optimal dialogue policies that maximize attack success
across multiple turns.
68df64e7df49fb0df1e03da3
Ruohao Guo
claimed_verified
2025-10-03T12:28:48.360Z
628711eb40423ef48fbc07af
https://cdn-avatars.huggingface.co/v1/production/uploads/628711eb40423ef48fbc07af/QmW1ow6LFNxPa0TyYBAJY.jpeg
Ruohao Guo
user
ruohao
68df64e7df49fb0df1e03da4
Afshin Oroojlooy
68df64e7df49fb0df1e03da5
Roshan Sridhar
68df64e7df49fb0df1e03da6
Miguel Ballesteros
68df64e7df49fb0df1e03da7
Alan Ritter
68df64e7df49fb0df1e03da8
Dan Roth
Tree-based Dialogue Reinforced Policy Optimization for Red-Teaming
  Attacks
Despite recent rapid progress in AI safety, current large language models
remain vulnerable to adversarial attacks in multi-turn interaction settings,
where attackers strategically adapt their prompts across conversation turns and
pose a more critical yet realistic challenge. Existing approaches that discover
safety vulnerabilities either rely on manual red-teaming with human experts or
employ automated methods using pre-defined templates and human-curated attack
data, with most focusing on single-turn attacks. However, these methods did not
explore the vast space of possible multi-turn attacks, failing to consider
novel attack trajectories that emerge from complex dialogue dynamics and
strategic conversation planning. This gap is particularly critical given recent
findings that LLMs exhibit significantly higher vulnerability to multi-turn
attacks compared to single-turn attacks. We propose DialTree-RPO, an on-policy
reinforcement learning framework integrated with tree search that autonomously
discovers diverse multi-turn attack strategies by treating the dialogue as a
sequential decision-making problem, enabling systematic exploration without
manually curated data. Through extensive experiments, our approach not only
achieves more than 25.9% higher ASR across 10 target models compared to
previous state-of-the-art approaches, but also effectively uncovers new attack
strategies by learning optimal dialogue policies that maximize attack success
across multiple turns.
DialTree-RPO, an on-policy reinforcement learning framework with tree search, autonomously discovers diverse multi-turn attack strategies against large language models, achieving higher attack success rates and uncovering new attack trajectories.
adversarial attacks multi-turn interaction reinforcement learning tree search sequential decision-making dialogue policies attack success rate