Alignment Tipping Process: How Self-Evolution Pushes LLM Agents Off the
  Rails
As Large Language Model (LLM) agents increasingly gain self-evolutionary
capabilities to adapt and refine their strategies through real-world
interaction, their long-term reliability becomes a critical concern. We
identify the Alignment Tipping Process (ATP), a critical post-deployment risk
unique to self-evolving LLM agents. Unlike training-time failures, ATP arises
when continual interaction drives agents to abandon alignment constraints
established during training in favor of reinforced, self-interested strategies.
We formalize and analyze ATP through two complementary paradigms:
Self-Interested Exploration, where repeated high-reward deviations induce
individual behavioral drift, and Imitative Strategy Diffusion, where deviant
behaviors spread across multi-agent systems. Building on these paradigms, we
construct controllable testbeds and benchmark Qwen3-8B and
Llama-3.1-8B-Instruct. Our experiments show that alignment benefits erode
rapidly under self-evolution, with initially aligned models converging toward
unaligned states. In multi-agent settings, successful violations diffuse
quickly, leading to collective misalignment. Moreover, current reinforcement
learning-based alignment methods provide only fragile defenses against
alignment tipping. Together, these findings demonstrate that alignment of LLM
agents is not a static property but a fragile and dynamic one, vulnerable to
feedback-driven decay during deployment. Our data and code are available at
https://github.com/aiming-lab/ATP.
68e4a0f8e4e093a7044e4dd7
Siwei Han
68e4a0f8e4e093a7044e4dd8
Jiaqi Liu
68e4a0f8e4e093a7044e4dd9
Yaofeng Su
68e4a0f8e4e093a7044e4dda
Wenbo Duan
68e4a0f8e4e093a7044e4ddb
Xinyuan Liu
68e4a0f8e4e093a7044e4ddc
Cihang Xie
68e4a0f8e4e093a7044e4ddd
Mohit Bansal
68e4a0f8e4e093a7044e4dde
Mingyu Ding
68e4a0f8e4e093a7044e4ddf
Linjun Zhang
68e4a0f8e4e093a7044e4de0
Huaxiu Yao
Alignment Tipping Process: How Self-Evolution Pushes LLM Agents Off the
  Rails
As Large Language Model (LLM) agents increasingly gain self-evolutionary
capabilities to adapt and refine their strategies through real-world
interaction, their long-term reliability becomes a critical concern. We
identify the Alignment Tipping Process (ATP), a critical post-deployment risk
unique to self-evolving LLM agents. Unlike training-time failures, ATP arises
when continual interaction drives agents to abandon alignment constraints
established during training in favor of reinforced, self-interested strategies.
We formalize and analyze ATP through two complementary paradigms:
Self-Interested Exploration, where repeated high-reward deviations induce
individual behavioral drift, and Imitative Strategy Diffusion, where deviant
behaviors spread across multi-agent systems. Building on these paradigms, we
construct controllable testbeds and benchmark Qwen3-8B and
Llama-3.1-8B-Instruct. Our experiments show that alignment benefits erode
rapidly under self-evolution, with initially aligned models converging toward
unaligned states. In multi-agent settings, successful violations diffuse
quickly, leading to collective misalignment. Moreover, current reinforcement
learning-based alignment methods provide only fragile defenses against
alignment tipping. Together, these findings demonstrate that alignment of LLM
agents is not a static property but a fragile and dynamic one, vulnerable to
feedback-driven decay during deployment. Our data and code are available at
https://github.com/aiming-lab/ATP.
Self-evolving LLM agents can abandon alignment constraints post-deployment, leading to rapid misalignment and collective failure in multi-agent systems.
Alignment Tipping Process Self-Interested Exploration Imitative Strategy Diffusion reinforcement learning-based alignment methods