Stop Overthinking: A Survey on Efficient Reasoning for Large Language
  Models
Large Language Models (LLMs) have demonstrated remarkable capabilities in
complex tasks. Recent advancements in Large Reasoning Models (LRMs), such as
OpenAI o1 and DeepSeek-R1, have further improved performance in System-2
reasoning domains like mathematics and programming by harnessing supervised
fine-tuning (SFT) and reinforcement learning (RL) techniques to enhance the
Chain-of-Thought (CoT) reasoning. However, while longer CoT reasoning sequences
improve performance, they also introduce significant computational overhead due
to verbose and redundant outputs, known as the "overthinking phenomenon". In
this paper, we provide the first structured survey to systematically
investigate and explore the current progress toward achieving efficient
reasoning in LLMs. Overall, relying on the inherent mechanism of LLMs, we
categorize existing works into several key directions: (1) model-based
efficient reasoning, which considers optimizing full-length reasoning models
into more concise reasoning models or directly training efficient reasoning
models; (2) reasoning output-based efficient reasoning, which aims to
dynamically reduce reasoning steps and length during inference; (3) input
prompts-based efficient reasoning, which seeks to enhance reasoning efficiency
based on input prompt properties such as difficulty or length control.
Additionally, we introduce the use of efficient data for training reasoning
models, explore the reasoning capabilities of small language models, and
discuss evaluation methods and benchmarking.
67dcdbfc71027d42fa46e3f2
Yang Sui
claimed_verified
2025-03-21T11:40:51.244Z
63787b13500186f250ba377c
/avatars/96bb6051662109e9cd25e6df7d738f17.svg
yangsui
user
yangsui
67dcdbfc71027d42fa46e3f3
Yu-Neng Chuang
67dcdbfc71027d42fa46e3f4
Guanchu Wang
claimed_verified
2025-06-04T09:05:18.328Z
62a4bc35e0de0c5c6da33c99
/avatars/e07f50d16c5f50a606d3cb408df83dab.svg
Guanchu
user
guan-wang
67dcdbfc71027d42fa46e3f5
Jiamu Zhang
claimed_verified
2025-03-24T08:54:22.156Z
661cdb5622ca1dfd3ff70976
https://cdn-avatars.huggingface.co/v1/production/uploads/661cdb5622ca1dfd3ff70976/rAmRx6T-_a9n0hfw1hvND.jpeg
Jiamu 'Morris' Zhang
user
morriszjm
67dcdbfc71027d42fa46e3f6
Tianyi Zhang
67dcdbfc71027d42fa46e3f7
Jiayi Yuan
admin_assigned
2025-03-21T11:59:47.887Z
64b7879a6ab5d14ca7f9bdf4
/avatars/48990fe5b18ab17a3dedacdcc6ee3f3a.svg
Jiayi Yuan
user
jy-yuan
67dcdbfc71027d42fa46e3f8
Hongyi Liu
admin_assigned
2025-03-21T11:59:56.207Z
65c540e2735dd9c81625fadd
/avatars/36713db387abb001cc1e20644ff1f1d4.svg
Hongyi Liu
user
HongyiLiuAI
67dcdbfc71027d42fa46e3f9
Andrew Wen
admin_assigned
2025-03-21T12:00:02.782Z
658fb57da02954c9820eeb29
/avatars/57a3a9c5175d0b23cc7d6cab0fec08b7.svg
andrew wen
user
andrewwen
67dcdbfc71027d42fa46e3fa
Shaochen
67dcdbfc71027d42fa46e3fb
Zhong
67dcdbfc71027d42fa46e3fc
Hanjie Chen
67dcdbfc71027d42fa46e3fd
Xia Hu
Stop Overthinking: A Survey on Efficient Reasoning for Large Language
  Models
Large Language Models (LLMs) have demonstrated remarkable capabilities in
complex tasks. Recent advancements in Large Reasoning Models (LRMs), such as
OpenAI o1 and DeepSeek-R1, have further improved performance in System-2
reasoning domains like mathematics and programming by harnessing supervised
fine-tuning (SFT) and reinforcement learning (RL) techniques to enhance the
Chain-of-Thought (CoT) reasoning. However, while longer CoT reasoning sequences
improve performance, they also introduce significant computational overhead due
to verbose and redundant outputs, known as the "overthinking phenomenon". In
this paper, we provide the first structured survey to systematically
investigate and explore the current progress toward achieving efficient
reasoning in LLMs. Overall, relying on the inherent mechanism of LLMs, we
categorize existing works into several key directions: (1) model-based
efficient reasoning, which considers optimizing full-length reasoning models
into more concise reasoning models or directly training efficient reasoning
models; (2) reasoning output-based efficient reasoning, which aims to
dynamically reduce reasoning steps and length during inference; (3) input
prompts-based efficient reasoning, which seeks to enhance reasoning efficiency
based on input prompt properties such as difficulty or length control.
Additionally, we introduce the use of efficient data for training reasoning
models, explore the reasoning capabilities of small language models, and
discuss evaluation methods and benchmarking.
This survey examines progress and methods for improving efficient reasoning in Large Language Models (LLMs), focusing on model, output, and input prompt optimizations, as well as the use of efficient data and small models.
Large Language Models (LLMs) Large Reasoning Models (LRMs) supervised fine-tuning (SFT) reinforcement learning (RL) Chain-of-Thought (CoT) reasoning overthinking phenomenon efficient reasoning model-based efficient reasoning reasoning output-based efficient reasoning input prompts-based efficient reasoning efficient data small language models evaluation methods benchmarking