Beyond Components: Singular Vector-Based Interpretability of Transformer Circuits
areeb ahmad, Abhinav Joshi, Ashutosh Modi
In the quest for interpretability of LLMs, circuit discovery has emerged as a powerful framework. It identifies a computational subgraph of models that can replicate a model's behavior on a certain task. Existing methods operate at the standard component level granularity (e.g., attention heads, MLPs), potentially overlooking more fine-grained computational structure. We propose a framework that treats singular vector pairs of the augmented query–key, value–output, and MLP projection matrices as the atomic units of inspection of the model's behavior. By applying singular value decomposition (SVD) to the weight matrices of attention and MLP layers, we uncover orthogonal functional directions within each component that independently contribute to task behavior. These directions define a compositional basis over which distinct computations can occur in parallel, even within a single attention head. Our framework can replicate GPT-2 behavior faithfully on tasks like indirect object identification. We also show that some of these directions act as a control knob for concepts which corresponds to meaningful subtasks within tasks. Our approach establishes a more precise foundation for automated interpretability that better aligns with the underlying low-rank structure of the transformer weights.