Efficient Reasoning Models: A Survey
Reasoning models have demonstrated remarkable progress in solving complex and
logic-intensive tasks by generating extended Chain-of-Thoughts (CoTs) prior to
arriving at a final answer. Yet, the emergence of this "slow-thinking"
paradigm, with numerous tokens generated in sequence, inevitably introduces
substantial computational overhead. To this end, it highlights an urgent need
for effective acceleration. This survey aims to provide a comprehensive
overview of recent advances in efficient reasoning. It categorizes existing
works into three key directions: (1) shorter - compressing lengthy CoTs into
concise yet effective reasoning chains; (2) smaller - developing compact
language models with strong reasoning capabilities through techniques such as
knowledge distillation, other model compression techniques, and reinforcement
learning; and (3) faster - designing efficient decoding strategies to
accelerate inference. A curated collection of papers discussed in this survey
is available in our GitHub repository.
67ff8a2b81740ab021196e98
Sicheng Feng
claimed_verified
2025-04-16T11:57:54.189Z
67a4a26d5e65aa63c6d30e68
https://cdn-avatars.huggingface.co/v1/production/uploads/67a4a26d5e65aa63c6d30e68/GtodlJGw-_IL2DTXQTucz.jpeg
Sicheng Feng
user
FSCCS
67ff8a2b81740ab021196e99
Gongfan Fang
claimed_verified
2025-04-16T13:42:05.338Z
646a1939c37ca1e12308fe81
/avatars/752e9d86018e7d33ad8bcd741203fd86.svg
Gongfan Fang
user
Vinnnf
67ff8a2b81740ab021196e9a
Xinyin Ma
67ff8a2b81740ab021196e9b
Xinchao Wang
Efficient Reasoning Models: A Survey
Reasoning models have demonstrated remarkable progress in solving complex and
logic-intensive tasks by generating extended Chain-of-Thoughts (CoTs) prior to
arriving at a final answer. Yet, the emergence of this "slow-thinking"
paradigm, with numerous tokens generated in sequence, inevitably introduces
substantial computational overhead. To this end, it highlights an urgent need
for effective acceleration. This survey aims to provide a comprehensive
overview of recent advances in efficient reasoning. It categorizes existing
works into three key directions: (1) shorter - compressing lengthy CoTs into
concise yet effective reasoning chains; (2) smaller - developing compact
language models with strong reasoning capabilities through techniques such as
knowledge distillation, other model compression techniques, and reinforcement
learning; and (3) faster - designing efficient decoding strategies to
accelerate inference. A curated collection of papers discussed in this survey
is available in our GitHub repository.
The survey discusses methods to accelerate reasoning models by compressing Chain-of-Thoughts, developing compact models, and designing efficient decoding strategies.
reasoning models Chain-of-Thoughts (CoTs) knowledge distillation model compression techniques reinforcement learning efficient decoding strategies