Why Do Some Language Models Fake Alignment While Others Don't?
Abhay Sheshadri, John Hughes, Julian Michael, Alex Mallen, Arun Jose, Fabien Roger
*Alignment faking in large language models* presented a demonstration of Claude 3 Opus and Claude 3.5 Sonnet selectively complying with a helpful-only training objective to prevent modification of their behavior outside of training. We expand this analysis to 23 models and find that only 5 (Claude 3 Opus, Claude 3.5 Sonnet, Llama 3 405B, Grok 3, Gemini 2.0 Flash) comply with harmful queries more when they infer they are in training than when they infer they are in deployment. First, we study the motivations of these 5 models and find surprising differences in the motivations that result in this compliance gap. Second, we investigate why models like GPT-4o or DeepSeek-V3 don’t fake alignment. Our results suggest this is not due to a lack of capabilities: we find that base models of GPT-4 and DeepSeek-V3 fake alignment, and that models fine-tuned to refuse less and pay more attention to details of the scenario also fake alignment. Our results indicate that variations in refusal behavior may account for a significant portion of differences in alignment faking, which suggests that post-training methods may reduce alignment faking.