Don't Overthink it. Preferring Shorter Thinking Chains for Improved LLM
  Reasoning
Reasoning large language models (LLMs) heavily rely on scaling test-time
compute to perform complex reasoning tasks by generating extensive "thinking"
chains. While demonstrating impressive results, this approach incurs
significant computational costs and inference time. In this work, we challenge
the assumption that long thinking chains results in better reasoning
capabilities. We first demonstrate that shorter reasoning chains within
individual questions are significantly more likely to yield correct answers -
up to 34.5% more accurate than the longest chain sampled for the same question.
Based on these results, we suggest short-m@k, a novel reasoning LLM inference
method. Our method executes k independent generations in parallel and halts
computation once the first m thinking processes are done. The final answer is
chosen using majority voting among these m chains. Basic short-1@k demonstrates
similar or even superior performance over standard majority voting in
low-compute settings - using up to 40% fewer thinking tokens. short-3@k, while
slightly less efficient than short-1@k, consistently surpasses majority voting
across all compute budgets, while still being substantially faster (up to 33%
wall time reduction). Inspired by our results, we finetune an LLM using short,
long, and randomly selected reasoning chains. We then observe that training on
the shorter ones leads to better performance. Our findings suggest rethinking
current methods of test-time compute in reasoning LLMs, emphasizing that longer
"thinking" does not necessarily translate to improved performance and can,
counter-intuitively, lead to degraded results.
68368b76b399c7d3af071167
Michael Hassid
admin_assigned
2025-05-28T12:16:46.857Z
6547411a9295970f878aa52e
/avatars/6e240f0add27bf1a6c04a9618eccdf83.svg
Michael Hassid
user
hassid
68368b76b399c7d3af071168
Gabriel Synnaeve
admin_assigned
2025-05-28T12:16:53.157Z
630eac7931970d1cd4fbacf2
/avatars/b7ccbddfa745db854dc342be1327cd53.svg
Gabriel Synnaeve
user
gsynnaeve
68368b76b399c7d3af071169
Yossi Adi
admin_assigned
2025-05-28T12:16:59.326Z
6481e135578646b5c2386728
https://cdn-avatars.huggingface.co/v1/production/uploads/6481e135578646b5c2386728/SPva4iNw0pORiCXD45cx9.jpeg
Yossi Adi
user
adiyoss
68368b76b399c7d3af07116a
Roy Schwartz
Don't Overthink it. Preferring Shorter Thinking Chains for Improved LLM
  Reasoning
Reasoning large language models (LLMs) heavily rely on scaling test-time
compute to perform complex reasoning tasks by generating extensive "thinking"
chains. While demonstrating impressive results, this approach incurs
significant computational costs and inference time. In this work, we challenge
the assumption that long thinking chains results in better reasoning
capabilities. We first demonstrate that shorter reasoning chains within
individual questions are significantly more likely to yield correct answers -
up to 34.5% more accurate than the longest chain sampled for the same question.
Based on these results, we suggest short-m@k, a novel reasoning LLM inference
method. Our method executes k independent generations in parallel and halts
computation once the first m thinking processes are done. The final answer is
chosen using majority voting among these m chains. Basic short-1@k demonstrates
similar or even superior performance over standard majority voting in
low-compute settings - using up to 40% fewer thinking tokens. short-3@k, while
slightly less efficient than short-1@k, consistently surpasses majority voting
across all compute budgets, while still being substantially faster (up to 33%
wall time reduction). Inspired by our results, we finetune an LLM using short,
long, and randomly selected reasoning chains. We then observe that training on
the shorter ones leads to better performance. Our findings suggest rethinking
current methods of test-time compute in reasoning LLMs, emphasizing that longer
"thinking" does not necessarily translate to improved performance and can,
counter-intuitively, lead to degraded results.
Shorter reasoning chains in LLMs can achieve similar or better performance with reduced computational cost and inference time compared to longer chains.
reasoning large language models thinking chains majority voting compute budgets wall time reduction