Refusal Falls off a Cliff: How Safety Alignment Fails in Reasoning?
Large reasoning models (LRMs) with multi-step reasoning capabilities have
shown remarkable problem-solving abilities, yet they exhibit concerning safety
vulnerabilities that remain poorly understood. In this work, we investigate why
safety alignment fails in reasoning models through a mechanistic
interpretability lens. Using a linear probing approach to trace refusal
intentions across token positions, we discover a striking phenomenon termed as
refusal cliff: many poorly-aligned reasoning models correctly identify
harmful prompts and maintain strong refusal intentions during their thinking
process, but experience a sharp drop in refusal scores at the final tokens
before output generation. This suggests that these models are not inherently
unsafe; rather, their refusal intentions are systematically suppressed. Through
causal intervention analysis, we identify a sparse set of attention heads that
negatively contribute to refusal behavior. Ablating just 3\% of these heads can
reduce attack success rates below 10\%. Building on these mechanistic insights,
we propose Cliff-as-a-Judge, a novel data selection method that
identifies training examples exhibiting the largest refusal cliff to
efficiently repair reasoning models' safety alignment. This approach achieves
comparable safety improvements using only 1.7\% of the vanilla safety training
data, demonstrating a less-is-more effect in safety alignment.
68e5ee60975ac4c405ef2167
Qingyu Yin
68e5ee60975ac4c405ef2168
Chak Tou Leong
68e5ee60975ac4c405ef2169
Linyi Yang
claimed_verified
2025-10-08T05:49:08.431Z
64895683f534abe18eec264b
/avatars/73cc9e6db6db86793787750776b57c63.svg
Linyi Yang
user
linyiyang2023
68e5ee60975ac4c405ef216a
Wenxuan Huang
claimed_verified
2025-10-11T14:05:27.192Z
67dc162ec8c00778e8689f42
https://cdn-avatars.huggingface.co/v1/production/uploads/67dc162ec8c00778e8689f42/_y_tO6W3ONOkOWbumAFXA.png
Wenxuan Huang
user
Osilly
68e5ee60975ac4c405ef216b
Wenjie Li
68e5ee60975ac4c405ef216c
Xiting Wang
68e5ee60975ac4c405ef216d
Jaehong Yoon
68e5ee60975ac4c405ef216e
YunXing
68e5ee60975ac4c405ef216f
XingYu
68e5ee60975ac4c405ef2170
Jinjin Gu
Refusal Falls off a Cliff: How Safety Alignment Fails in Reasoning?
Large reasoning models (LRMs) with multi-step reasoning capabilities have
shown remarkable problem-solving abilities, yet they exhibit concerning safety
vulnerabilities that remain poorly understood. In this work, we investigate why
safety alignment fails in reasoning models through a mechanistic
interpretability lens. Using a linear probing approach to trace refusal
intentions across token positions, we discover a striking phenomenon termed as
refusal cliff: many poorly-aligned reasoning models correctly identify
harmful prompts and maintain strong refusal intentions during their thinking
process, but experience a sharp drop in refusal scores at the final tokens
before output generation. This suggests that these models are not inherently
unsafe; rather, their refusal intentions are systematically suppressed. Through
causal intervention analysis, we identify a sparse set of attention heads that
negatively contribute to refusal behavior. Ablating just 3\% of these heads can
reduce attack success rates below 10\%. Building on these mechanistic insights,
we propose Cliff-as-a-Judge, a novel data selection method that
identifies training examples exhibiting the largest refusal cliff to
efficiently repair reasoning models' safety alignment. This approach achieves
comparable safety improvements using only 1.7\% of the vanilla safety training
data, demonstrating a less-is-more effect in safety alignment.
Research identifies a mechanism called the refusal cliff in large reasoning models, where refusal intentions drop sharply before output generation, and proposes a method to improve safety by focusing on specific attention heads and training examples.
large reasoning models multi-step reasoning safety vulnerabilities mechanistic interpretability linear probing refusal cliff token positions refusal intentions causal intervention analysis attention heads Cliff-as-a-Judge data selection method safety alignment