Adaptive Correction During the LLM Forward Pass
Arjun Patrawala, Jiahai Feng, Erik Jones, Jacob Steinhardt
Recent methods in language model interpretability employ sparse autoencoders and other techniques to decompose residual stream contributions into linear, semantically-meaningful features. Our work demonstrates that the underlying assumption of these methods—that residual stream contributions build additively upon each other—is fundamentally incomplete. We identify the Transformer Layer Correction Mechanism (TLCM), wherein adjacent transformer layers systematically counteract each other's contributions to the residual stream. TLCM appears in 5 out of 7 major open-source model families and operates across diverse contexts. We establish that TLCM emerges during pretraining, operates most strongly on tokens with high contextual dependency, and adaptively calibrates its correction strength based on the preceding layer's output. We further demonstrate that TLCM functions by selectively promoting and rejecting specific subspaces. We propose a "propose-and-reject" hypothesis: layers may suggest multiple potential features, while subsequent layers selectively filter inappropriate ones. This framework explains three persistent challenges in feature-based interpretability: why extracted features descriptions often suffer from low specificity; why feature-based interventions for model steering are unsuccessful at low magnitude; why recent work finds cross-layer transcoders outperform SAEs.