LLMs Learn to Deceive Unintentionally: Emergent Misalignment in
  Dishonesty from Misaligned Samples to Biased Human-AI Interactions
Previous research has shown that LLMs finetuned on malicious or incorrect
completions within narrow domains (e.g., insecure code or incorrect medical
advice) can become broadly misaligned to exhibit harmful behaviors, which is
called emergent misalignment. In this work, we investigate whether this
phenomenon can extend beyond safety behaviors to a broader spectrum of
dishonesty and deception under high-stakes scenarios (e.g., lying under
pressure and deceptive behavior). To explore this, we finetune open-sourced
LLMs on misaligned completions across diverse domains. Experimental results
demonstrate that LLMs show broadly misaligned behavior in dishonesty.
Additionally, we further explore this phenomenon in a downstream combined
finetuning setting, and find that introducing as little as 1% of misalignment
data into a standard downstream task is sufficient to decrease honest behavior
over 20%. Furthermore, we consider a more practical human-AI interaction
environment where we simulate both benign and biased users to interact with the
assistant LLM. Notably, we find that the assistant can be misaligned
unintentionally to exacerbate its dishonesty with only 10% biased user
population. In summary, we extend the study of emergent misalignment to the
domain of dishonesty and deception under high-stakes scenarios, and demonstrate
that this risk arises not only through direct finetuning, but also in
downstream mixture tasks and practical human-AI interactions.
68e8839f95e8e6771df38a70
XuHao Hu
claimed_verified
2025-10-10T04:31:51.718Z
6372813520a58a5e14c596a3
/avatars/9135151259db3e5b9c8969e1d00c949d.svg
XuHao Hu
user
Foreshhh
68e8839f95e8e6771df38a71
Peng Wang
68e8839f95e8e6771df38a72
Xiaoya Lu
68e8839f95e8e6771df38a73
Dongrui Liu
68e8839f95e8e6771df38a74
Xuanjing Huang
68e8839f95e8e6771df38a75
Jing Shao
LLMs Learn to Deceive Unintentionally: Emergent Misalignment in
  Dishonesty from Misaligned Samples to Biased Human-AI Interactions
Previous research has shown that LLMs finetuned on malicious or incorrect
completions within narrow domains (e.g., insecure code or incorrect medical
advice) can become broadly misaligned to exhibit harmful behaviors, which is
called emergent misalignment. In this work, we investigate whether this
phenomenon can extend beyond safety behaviors to a broader spectrum of
dishonesty and deception under high-stakes scenarios (e.g., lying under
pressure and deceptive behavior). To explore this, we finetune open-sourced
LLMs on misaligned completions across diverse domains. Experimental results
demonstrate that LLMs show broadly misaligned behavior in dishonesty.
Additionally, we further explore this phenomenon in a downstream combined
finetuning setting, and find that introducing as little as 1% of misalignment
data into a standard downstream task is sufficient to decrease honest behavior
over 20%. Furthermore, we consider a more practical human-AI interaction
environment where we simulate both benign and biased users to interact with the
assistant LLM. Notably, we find that the assistant can be misaligned
unintentionally to exacerbate its dishonesty with only 10% biased user
population. In summary, we extend the study of emergent misalignment to the
domain of dishonesty and deception under high-stakes scenarios, and demonstrate
that this risk arises not only through direct finetuning, but also in
downstream mixture tasks and practical human-AI interactions.
LLMs finetuned on misaligned data exhibit dishonest behavior, which can be exacerbated in downstream tasks and human-AI interactions.
LLMs finetuning emergent misalignment dishonesty deception high-stakes scenarios downstream tasks human-AI interaction biased users