Thinking Sparks!: Emergent Attention Heads in Reasoning Models During
  Post Training
The remarkable capabilities of modern large reasoning models are largely
unlocked through post-training techniques such as supervised fine-tuning and
reinforcement learning. However, the architectural mechanisms behind such
improvements remain largely opaque. In this work, we use circuit analysis to
demonstrate that post-training for complex reasoning sparks the emergence of
novel, functionally specialized attention heads. These heads collectively
support structured reasoning and computation. Our comparative analysis across
Qwen families and DeepSeek-distilled model reveals that these emergent heads
evolve differently under different training regimes. Distillation and SFT
foster a cumulative addition of stable reasoning heads. In contrast, group
relative policy optimization operates in a dynamic search mode: relatively few
attention heads are iteratively activated, evaluated, and pruned, with their
survival closely tracking fluctuations in the task reward signal. Furthermore,
we find that controllable think on/off models do not possess dedicated thinking
heads. Instead, turning off explicit reasoning triggers a broader-but less
efficient-set of compensatory heads. Through ablation and qualitative analyses,
we connect these circuit-level dynamics to a crucial performance trade-off:
strengthened heads enable sophisticated problem-solving strategies for
difficult problems but can also introduce over-thinking failure modes, such as
calculation errors or logical loops on simpler tasks. These findings connect
circuit-level dynamics to macro-level performance, identifying an inherent
tension where complex reasoning comes at the cost of elementary computations.
More broadly, our work points to future directions for training policy design,
emphasizing the need to balance the development of effective reasoning
strategies with the assurance of reliable, flawless execution.
68dc8a1f4159d1f2418f9a64
Yein Park
68dc8a1f4159d1f2418f9a65
Minbyul Jeong
68dc8a1f4159d1f2418f9a66
Jaewoo Kang
Thinking Sparks!: Emergent Attention Heads in Reasoning Models During
  Post Training
The remarkable capabilities of modern large reasoning models are largely
unlocked through post-training techniques such as supervised fine-tuning and
reinforcement learning. However, the architectural mechanisms behind such
improvements remain largely opaque. In this work, we use circuit analysis to
demonstrate that post-training for complex reasoning sparks the emergence of
novel, functionally specialized attention heads. These heads collectively
support structured reasoning and computation. Our comparative analysis across
Qwen families and DeepSeek-distilled model reveals that these emergent heads
evolve differently under different training regimes. Distillation and SFT
foster a cumulative addition of stable reasoning heads. In contrast, group
relative policy optimization operates in a dynamic search mode: relatively few
attention heads are iteratively activated, evaluated, and pruned, with their
survival closely tracking fluctuations in the task reward signal. Furthermore,
we find that controllable think on/off models do not possess dedicated thinking
heads. Instead, turning off explicit reasoning triggers a broader-but less
efficient-set of compensatory heads. Through ablation and qualitative analyses,
we connect these circuit-level dynamics to a crucial performance trade-off:
strengthened heads enable sophisticated problem-solving strategies for
difficult problems but can also introduce over-thinking failure modes, such as
calculation errors or logical loops on simpler tasks. These findings connect
circuit-level dynamics to macro-level performance, identifying an inherent
tension where complex reasoning comes at the cost of elementary computations.
More broadly, our work points to future directions for training policy design,
emphasizing the need to balance the development of effective reasoning
strategies with the assurance of reliable, flawless execution.
Post-training techniques like supervised fine-tuning and reinforcement learning lead to the emergence of specialized attention heads that support structured reasoning, with different training regimes affecting their evolution and performance.
supervised fine-tuning reinforcement learning circuit analysis attention heads structured reasoning Qwen families DeepSeek-distilled model group relative policy optimization think on/off models ablation analysis qualitative analysis over-thinking failure modes