The Rogue Scalpel: Activation Steering Compromises LLM Safety
Activation steering is a promising technique for controlling LLM behavior by
adding semantically meaningful vectors directly into a model's hidden states
during inference. It is often framed as a precise, interpretable, and
potentially safer alternative to fine-tuning. We demonstrate the opposite:
steering systematically breaks model alignment safeguards, making it comply
with harmful requests. Through extensive experiments on different model
families, we show that even steering in a random direction can increase the
probability of harmful compliance from 0% to 2-27%. Alarmingly, steering benign
features from a sparse autoencoder (SAE), a common source of interpretable
directions, increases these rates by a further 2-4%. Finally, we show that
combining 20 randomly sampled vectors that jailbreak a single prompt creates a
universal attack, significantly increasing harmful compliance on unseen
requests. These results challenge the paradigm of safety through
interpretability, showing that precise control over model internals does not
guarantee precise control over model behavior.
68dec5ae70ada21878c75169
Anton Korznikov
68dec5ae70ada21878c7516a
Andrey Galichin
68dec5ae70ada21878c7516b
Alexey Dontsov
claimed_verified
2025-10-03T12:33:28.616Z
60cd95ee15ecba5f2200304a
https://cdn-avatars.huggingface.co/v1/production/uploads/60cd95ee15ecba5f2200304a/3gMYeWm8wQO5KfqE5RmEe.jpeg
Alexey Dontsov
user
therem
68dec5ae70ada21878c7516c
Oleg Y. Rogov
68dec5ae70ada21878c7516d
Ivan Oseledets
68dec5ae70ada21878c7516e
Elena Tutubalina
claimed_verified
2025-10-03T12:33:26.134Z
662f8d645c4db70c77a203b0
/avatars/72f9a3c39b3ba5114388d16a35524835.svg
Elena Tutubalina
user
tlenusik
The Rogue Scalpel: Activation Steering Compromises LLM Safety
Activation steering is a promising technique for controlling LLM behavior by
adding semantically meaningful vectors directly into a model's hidden states
during inference. It is often framed as a precise, interpretable, and
potentially safer alternative to fine-tuning. We demonstrate the opposite:
steering systematically breaks model alignment safeguards, making it comply
with harmful requests. Through extensive experiments on different model
families, we show that even steering in a random direction can increase the
probability of harmful compliance from 0% to 2-27%. Alarmingly, steering benign
features from a sparse autoencoder (SAE), a common source of interpretable
directions, increases these rates by a further 2-4%. Finally, we show that
combining 20 randomly sampled vectors that jailbreak a single prompt creates a
universal attack, significantly increasing harmful compliance on unseen
requests. These results challenge the paradigm of safety through
interpretability, showing that precise control over model internals does not
guarantee precise control over model behavior.
Activation steering, intended to control LLM behavior, can instead increase harmful compliance and undermine model alignment safeguards.
activation steering LLM behavior semantically meaningful vectors hidden states model alignment safeguards harmful requests sparse autoencoder universal attack