When Does Reasoning Matter? A Controlled Study of Reasoning's
  Contribution to Model Performance
Large Language Models (LLMs) with reasoning capabilities have achieved
state-of-the-art performance on a wide range of tasks. Despite its empirical
success, the tasks and model scales at which reasoning becomes effective, as
well as its training and inference costs, remain underexplored. In this work,
we rely on a synthetic data distillation framework to conduct a large-scale
supervised study. We compare Instruction Fine-Tuning (IFT) and reasoning models
of varying sizes, on a wide range of math-centric and general-purpose tasks,
evaluating both multiple-choice and open-ended formats. Our analysis reveals
that reasoning consistently improves model performance, often matching or
surpassing significantly larger IFT systems. Notably, while IFT remains
Pareto-optimal in training and inference costs, reasoning models become
increasingly valuable as model size scales, overcoming IFT performance limits
on reasoning-intensive and open-ended tasks.
68da3f770177a6054b013c5a
Nicolas Boizard
claimed_verified
2025-09-30T06:36:23.593Z
62be186a5f59ff2320e6e32b
https://cdn-avatars.huggingface.co/v1/production/uploads/62be186a5f59ff2320e6e32b/W_emoC2uItM-MJZyCfIKI.png
Nicolas-BZRD
user
Nicolas-BZRD
68da3f770177a6054b013c5b
Hippolyte Gisserot-Boukhlef
68da3f770177a6054b013c5c
Kevin El-Haddad
68da3f770177a6054b013c5d
Céline Hudelot
68da3f770177a6054b013c5e
Pierre Colombo
When Does Reasoning Matter? A Controlled Study of Reasoning's
  Contribution to Model Performance
Large Language Models (LLMs) with reasoning capabilities have achieved
state-of-the-art performance on a wide range of tasks. Despite its empirical
success, the tasks and model scales at which reasoning becomes effective, as
well as its training and inference costs, remain underexplored. In this work,
we rely on a synthetic data distillation framework to conduct a large-scale
supervised study. We compare Instruction Fine-Tuning (IFT) and reasoning models
of varying sizes, on a wide range of math-centric and general-purpose tasks,
evaluating both multiple-choice and open-ended formats. Our analysis reveals
that reasoning consistently improves model performance, often matching or
surpassing significantly larger IFT systems. Notably, while IFT remains
Pareto-optimal in training and inference costs, reasoning models become
increasingly valuable as model size scales, overcoming IFT performance limits
on reasoning-intensive and open-ended tasks.
Reasoning models enhance performance across various tasks, surpassing instruction fine-tuned models in reasoning-intensive and open-ended tasks, despite higher computational costs.
Instruction Fine-Tuning reasoning models synthetic data distillation multiple-choice open-ended formats reasoning-intensive tasks