Reasoning Vectors: Transferring Chain-of-Thought Capabilities via Task
  Arithmetic
Large language models often require costly optimization, such as
reinforcement learning, to master complex reasoning tasks. This work
demonstrates that reasoning ability, once learned, can be extracted and
transferred between models as a compact task vector. We source two publicly
available, identically initialized Qwen2.5 models, one fine-tuned with
supervised fine-tuning (SFT) and the other with group relative policy
optimization (GRPO) on the same dataset. From these, we extract a reasoning
vector: v_{reason} = theta_{GRPO} - theta_{SFT}. We
hypothesize that this vector captures the reasoning capability instilled by
reinforcement learning while factoring out shared knowledge from the SFT
process. When added to compatible instruction-tuned models through simple
arithmetic, this vector consistently improves performance across diverse
reasoning benchmarks: GSM8K (+4.9%), HumanEval (+4.3%), SciQ (+1.7%), and
BigBenchHard (+12.3% for the 1.5B model). The performance improvements persist
under adversarial conditions. Conversely, subtracting the vector causes
significant performance degradation (-11.8% on GSM8K), demonstrating the
vector's strong contribution to the model's reasoning abilities. This work
shows how reasoning capabilities, typically developed through expensive
training, can be extracted from existing open-source models and reused through
simple tensor arithmetic, offering a practical way to enhance models by
recycling prior computational investments.
68b7e1c3295f15ff609114cd
Mohammad Zbeeb
claimed_verified
2025-09-03T19:25:13.927Z
665ee96b9f29909d03abfa37
https://cdn-avatars.huggingface.co/v1/production/uploads/665ee96b9f29909d03abfa37/ipctrsDzM-3_s8GTzWsd2.png
Mohammad Zbeeb
user
zbeeb
68b7e1c3295f15ff609114ce
Hasan Abed Al Kader Hammoud
admin_assigned
2025-09-03T08:35:44.317Z
642b51385bf2355d02a23d15
/avatars/87985347643b2647555f2453fa4d94fb.svg
Hasan Abed Al Kader Hammoud
user
hammh0a
68b7e1c3295f15ff609114cf
Bernard Ghanem
Reasoning Vectors: Transferring Chain-of-Thought Capabilities via Task
  Arithmetic
Large language models often require costly optimization, such as
reinforcement learning, to master complex reasoning tasks. This work
demonstrates that reasoning ability, once learned, can be extracted and
transferred between models as a compact task vector. We source two publicly
available, identically initialized Qwen2.5 models, one fine-tuned with
supervised fine-tuning (SFT) and the other with group relative policy
optimization (GRPO) on the same dataset. From these, we extract a reasoning
vector: v_{reason} = theta_{GRPO} - theta_{SFT}. We
hypothesize that this vector captures the reasoning capability instilled by
reinforcement learning while factoring out shared knowledge from the SFT
process. When added to compatible instruction-tuned models through simple
arithmetic, this vector consistently improves performance across diverse
reasoning benchmarks: GSM8K (+4.9%), HumanEval (+4.3%), SciQ (+1.7%), and
BigBenchHard (+12.3% for the 1.5B model). The performance improvements persist
under adversarial conditions. Conversely, subtracting the vector causes
significant performance degradation (-11.8% on GSM8K), demonstrating the
vector's strong contribution to the model's reasoning abilities. This work
shows how reasoning capabilities, typically developed through expensive
training, can be extracted from existing open-source models and reused through
simple tensor arithmetic, offering a practical way to enhance models by
recycling prior computational investments.
Reasoning capabilities from reinforcement learning can be extracted as a task vector and transferred to other models to improve performance on diverse benchmarks.
reinforcement learning supervised fine-tuning group relative policy optimization reasoning vector instruction-tuned models GSM8K HumanEval SciQ BigBenchHard adversarial conditions