Towards Large Reasoning Models: A Survey of Reinforced Reasoning with
  Large Language Models
Language has long been conceived as an essential tool for human reasoning.
The breakthrough of Large Language Models (LLMs) has sparked significant
research interest in leveraging these models to tackle complex reasoning tasks.
Researchers have moved beyond simple autoregressive token generation by
introducing the concept of "thought" -- a sequence of tokens representing
intermediate steps in the reasoning process. This innovative paradigm enables
LLMs' to mimic complex human reasoning processes, such as tree search and
reflective thinking. Recently, an emerging trend of learning to reason has
applied reinforcement learning (RL) to train LLMs to master reasoning
processes. This approach enables the automatic generation of high-quality
reasoning trajectories through trial-and-error search algorithms, significantly
expanding LLMs' reasoning capacity by providing substantially more training
data. Furthermore, recent studies demonstrate that encouraging LLMs to "think"
with more tokens during test-time inference can further significantly boost
reasoning accuracy. Therefore, the train-time and test-time scaling combined to
show a new research frontier -- a path toward Large Reasoning Model. The
introduction of OpenAI's o1 series marks a significant milestone in this
research direction. In this survey, we present a comprehensive review of recent
progress in LLM reasoning. We begin by introducing the foundational background
of LLMs and then explore the key technical components driving the development
of large reasoning models, with a focus on automated data construction,
learning-to-reason techniques, and test-time scaling. We also analyze popular
open-source projects at building large reasoning models, and conclude with open
challenges and future research directions.
6789e04652739943c940af38
Fengli Xu
6789e04652739943c940af39
Qianyue Hao
admin_assigned
2025-01-17T10:42:15.441Z
6583e2b283a9e1460c6fb1e0
/avatars/a949165b1cec5e1d1d55f3af98182156.svg
Qianyue Hao
user
haohao11
6789e04652739943c940af3a
Zefang Zong
admin_assigned
2025-01-17T10:42:26.065Z
64feba7efa64465422ce3003
/avatars/abdc7a3748f6a4e15ebc6aa8d616c87d.svg
zongzefang
user
zzfoutofspace
6789e04652739943c940af3b
Jingwei Wang
claimed_verified
2025-01-30T08:41:20.675Z
66e59ff27d43e55cfd31bec9
/avatars/e046b51bf2a530255d3fb707991546a4.svg
Chen Gao
user
chgao96
6789e04652739943c940af3c
Yunke Zhang
admin_assigned
2025-01-17T10:43:05.430Z
64071cbc2e309e65451f87b6
/avatars/2201244380d221b9db5661f20510d853.svg
yunke zhang
user
berserkerko
6789e04652739943c940af3d
Jingyi Wang
6789e04652739943c940af3e
Xiaochong Lan
6789e04652739943c940af3f
Jiahui Gong
admin_assigned
2025-01-17T10:43:44.415Z
6509a040d95f30b9dcdbf789
/avatars/4e4b441e22a1f2c3387e5b981ba6bbbe.svg
Jiahui Gong
user
zhazhahui7
6789e04652739943c940af40
Tianjian Ouyang
admin_assigned
2025-01-17T10:43:50.599Z
6566fcb5118497d0af91dc3b
/avatars/e081d7d1b1657245cd818e5417cdcb2e.svg
TIANJIAN OUYANG
user
Ouyangtj
6789e04652739943c940af41
Fanjin Meng
6789e04652739943c940af42
Chenyang Shao
admin_assigned
2025-01-17T10:44:14.869Z
654dd731671d2c1ced0539e1
/avatars/7eaa5eee16aac9105cd2af7ee84841c3.svg
shaochenyang
user
l-1-l
6789e04652739943c940af43
Yuwei Yan
admin_assigned
2025-01-17T10:44:24.670Z
668e965959d9ffef7e8b035a
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/tqjvWWxXgTkNhYeNhoPJQ.jpeg
Yuwei Yan
user
PinkGranite
6789e04652739943c940af44
Qinglong Yang
admin_assigned
2025-01-17T10:44:31.561Z
663ef727f9c2d3c9da3b8d4b
/avatars/4bb431e410078f7ba2487597d1beb589.svg
qinglong yang
user
m912218831
6789e04652739943c940af45
Yiwen Song
admin_assigned
2025-01-17T10:44:57.348Z
6517326e335b4c728b3406a4
/avatars/cde4a672778fc69c172e6bdeee2d540a.svg
Yiwen Song
user
yiwen-song
6789e04652739943c940af46
Sijian Ren
6789e04652739943c940af47
Xinyuan Hu
6789e04652739943c940af48
Yu Li
6789e04652739943c940af49
Jie Feng
claimed_verified
2025-01-20T09:29:08.372Z
6465d3bd63e7e09dd02e95c3
/avatars/b2798bd5f8368f956bf7fab79d9432f0.svg
Jie Feng
user
JJ-TMT
6789e04652739943c940af4a
Chen Gao
admin_assigned
2025-01-17T10:45:59.888Z
65049f41c6ae3df8f28cfe96
/avatars/a653ed4a215c4bd34710a1cee9f4d9cd.svg
Chen Gao
user
gaochen315
6789e04652739943c940af4b
Yong Li
Towards Large Reasoning Models: A Survey of Reinforced Reasoning with
  Large Language Models
Language has long been conceived as an essential tool for human reasoning.
The breakthrough of Large Language Models (LLMs) has sparked significant
research interest in leveraging these models to tackle complex reasoning tasks.
Researchers have moved beyond simple autoregressive token generation by
introducing the concept of "thought" -- a sequence of tokens representing
intermediate steps in the reasoning process. This innovative paradigm enables
LLMs' to mimic complex human reasoning processes, such as tree search and
reflective thinking. Recently, an emerging trend of learning to reason has
applied reinforcement learning (RL) to train LLMs to master reasoning
processes. This approach enables the automatic generation of high-quality
reasoning trajectories through trial-and-error search algorithms, significantly
expanding LLMs' reasoning capacity by providing substantially more training
data. Furthermore, recent studies demonstrate that encouraging LLMs to "think"
with more tokens during test-time inference can further significantly boost
reasoning accuracy. Therefore, the train-time and test-time scaling combined to
show a new research frontier -- a path toward Large Reasoning Model. The
introduction of OpenAI's o1 series marks a significant milestone in this
research direction. In this survey, we present a comprehensive review of recent
progress in LLM reasoning. We begin by introducing the foundational background
of LLMs and then explore the key technical components driving the development
of large reasoning models, with a focus on automated data construction,
learning-to-reason techniques, and test-time scaling. We also analyze popular
open-source projects at building large reasoning models, and conclude with open
challenges and future research directions.
LLMs are enhanced with reasoning capabilities through token-based thoughts, reinforcement learning, and test-time scaling, representing a new frontier in large reasoning models.
Large Language Models LLMs autoregressive token generation thought tree search reflective thinking reinforcement learning reasoning trajectories trial-and-error search algorithms automated data construction learning-to-reason test-time scaling Large Reasoning Model OpenAI's o1 series