Decoupling Harmfulness from Refusal in LLMs
Jiachen Zhao, Jing Huang, Zhengxuan Wu, David Bau, Weiyan Shi
LLMs are trained to refuse harmful instructions, but do they truly understand harmfulness beyond just refusing? Prior work has shown that LLMs' refusal behavior is mediated by a one-dimensional subspace, i.e., a refusal direction, in the latent space. In this work, we identify a new dimension to analyze safety mechanisms in LLMs: we have found that harmfulness is encoded as a distinct concept from refusal in their latent representations. To disentangle these two safety signals, we construct a prompting task that inverts models' refusal response to acceptance and vice versa. We find that steering along the harmfulness direction causes LLMs to interpret harmless instructions as harmful, and reason about that increased harmfulness internally to fulfill the inversion task, while applying the refusal direction tends to directly amplify refusal signals without reversing the model’s harmfulness judgment. Furthermore, our clustering analysis of hidden states reveals that some jailbreak methods may not radically suppress the model's internal harmfulness judgment, but directly reduce refusal signals.  We also observe that narrow fine-tuning that reverses models' behaviors has minimal impact on the model's underlying beliefs about harmfulness and refusal. These insights lead to a practical application that latent harmfulness representations can serve as an intrinsic safeguard for detecting unsafe inputs and reducing misrefusals, which is also robust to fine-tuning attacks. For example, on Llama2, the latent harmfulness classifier can detect all the tested harmful prompts that jailbreak the model with adversarial suffixes.