Your Agent May Misevolve: Emergent Risks in Self-evolving LLM Agents
Advances in Large Language Models (LLMs) have enabled a new class of
self-evolving agents that autonomously improve through interaction with the
environment, demonstrating strong capabilities. However, self-evolution also
introduces novel risks overlooked by current safety research. In this work, we
study the case where an agent's self-evolution deviates in unintended ways,
leading to undesirable or even harmful outcomes. We refer to this as
Misevolution. To provide a systematic investigation, we evaluate misevolution
along four key evolutionary pathways: model, memory, tool, and workflow. Our
empirical findings reveal that misevolution is a widespread risk, affecting
agents built even on top-tier LLMs (e.g., Gemini-2.5-Pro). Different emergent
risks are observed in the self-evolutionary process, such as the degradation of
safety alignment after memory accumulation, or the unintended introduction of
vulnerabilities in tool creation and reuse. To our knowledge, this is the first
study to systematically conceptualize misevolution and provide empirical
evidence of its occurrence, highlighting an urgent need for new safety
paradigms for self-evolving agents. Finally, we discuss potential mitigation
strategies to inspire further research on building safer and more trustworthy
self-evolving agents. Our code and data are available at
https://github.com/ShaoShuai0605/Misevolution . Warning: this paper includes
examples that may be offensive or harmful in nature.
68dfe50f73e20ab5778419f0
Shuai Shao
68dfe50f73e20ab5778419f1
Qihan Ren
claimed_verified
2025-10-06T12:49:04.168Z
66e2624a436a1798365e4581
/avatars/6c605807d34faa8fb505e135a4b47776.svg
Qihan Ren
user
jasonrqh
68dfe50f73e20ab5778419f2
Chen Qian
68dfe50f73e20ab5778419f3
Boyi Wei
claimed_verified
2025-10-07T12:28:45.945Z
65319bd7f85995389d4f019c
/avatars/657858b8435b220c9a29918c0dae9c6d.svg
Boyi Wei
user
boyiwei
68dfe50f73e20ab5778419f4
Dadi Guo
68dfe50f73e20ab5778419f5
Jingyi Yang
claimed_verified
2025-10-05T12:46:26.913Z
64f73a44102fbfb26410962e
/avatars/328302a495de6a4418be835456d1d3c6.svg
jingyi Yang
user
JY-Young
68dfe50f73e20ab5778419f6
Xinhao Song
68dfe50f73e20ab5778419f7
Linfeng Zhang
68dfe50f73e20ab5778419f8
Weinan Zhang
68dfe50f73e20ab5778419f9
Dongrui Liu
68dfe50f73e20ab5778419fa
Jing Shao
Your Agent May Misevolve: Emergent Risks in Self-evolving LLM Agents
Advances in Large Language Models (LLMs) have enabled a new class of
self-evolving agents that autonomously improve through interaction with the
environment, demonstrating strong capabilities. However, self-evolution also
introduces novel risks overlooked by current safety research. In this work, we
study the case where an agent's self-evolution deviates in unintended ways,
leading to undesirable or even harmful outcomes. We refer to this as
Misevolution. To provide a systematic investigation, we evaluate misevolution
along four key evolutionary pathways: model, memory, tool, and workflow. Our
empirical findings reveal that misevolution is a widespread risk, affecting
agents built even on top-tier LLMs (e.g., Gemini-2.5-Pro). Different emergent
risks are observed in the self-evolutionary process, such as the degradation of
safety alignment after memory accumulation, or the unintended introduction of
vulnerabilities in tool creation and reuse. To our knowledge, this is the first
study to systematically conceptualize misevolution and provide empirical
evidence of its occurrence, highlighting an urgent need for new safety
paradigms for self-evolving agents. Finally, we discuss potential mitigation
strategies to inspire further research on building safer and more trustworthy
self-evolving agents. Our code and data are available at
https://github.com/ShaoShuai0605/Misevolution . Warning: this paper includes
examples that may be offensive or harmful in nature.
Self-evolving agents based on Large Language Models can deviate in unintended ways, leading to various risks such as safety misalignment and vulnerability introduction, necessitating new safety paradigms.
Large Language Models self-evolving agents Misevolution evolutionary pathways safety alignment memory accumulation tool creation vulnerability introduction