Light-R1: Curriculum SFT, DPO and RL for Long COT from Scratch and
  Beyond
This paper presents our work on the Light-R1 series, with models, data, and
code all released.
  We first focus on training long COT models from scratch, specifically
starting from models initially lacking long COT capabilities. Using a
curriculum training recipe consisting of two-stage SFT and semi-on-policy DPO,
we train our model Light-R1-32B from Qwen2.5-32B-Instruct, resulting in
superior math performance compared to DeepSeek-R1-Distill-Qwen-32B. Despite
being trained exclusively on math data, Light-R1-32B shows strong
generalization across other domains. In the subsequent phase of this work, we
highlight the significant benefit of the 3k dataset constructed for the second
SFT stage on enhancing other models. By fine-tuning DeepSeek-R1-Distilled
models using this dataset, we obtain new SOTA models in 7B and 14B, while the
32B model, Light-R1-32B-DS performed comparably to QwQ-32B and DeepSeek-R1.
  Furthermore, we extend our work by applying reinforcement learning,
specifically GRPO, on long-COT models to further improve reasoning performance.
We successfully train our final Light-R1-14B-DS with RL, achieving SOTA
performance among 14B parameter models in math. With AIME24 & 25 scores of 74.0
and 60.2 respectively, Light-R1-14B-DS surpasses even many 32B models and
DeepSeek-R1-Distill-Llama-70B. Its RL training also exhibits well expected
behavior, showing simultaneous increase in response length and reward score.
  The Light-R1 series of work validates training long-COT models from scratch,
showcases the art in SFT data and releases SOTA models from RL.
67d3a87df6ea55297c3cd071
Liang Wen
claimed_verified
2025-07-29T07:12:12.624Z
64eeb81ad0ceda46832e0160
/avatars/bbeb6a1582071dffcf3fca9780df5138.svg
wenliang
user
wenliang1990
67d3a87df6ea55297c3cd072
Yunke Cai
67d3a87df6ea55297c3cd073
Fenrui Xiao
admin_assigned
2025-03-14T11:07:18.770Z
643cbf3aae8d93dc395383e3
/avatars/3e719374cc14483e9cdfb174ab99ad26.svg
Fenrui Xiao
user
cizhenshi
67d3a87df6ea55297c3cd074
Xin He
67d3a87df6ea55297c3cd075
Qi An
67d3a87df6ea55297c3cd076
Zhenyu Duan
admin_assigned
2025-03-14T11:07:33.157Z
646c2f4d628e5b50b2e22511
/avatars/da64cfd3320f00a1718ab40165ed35dd.svg
zhenyu duan
user
duan901010
67d3a87df6ea55297c3cd077
Yimin Du
admin_assigned
2025-03-14T11:07:12.002Z
66fe5e256990f417165835e8
/avatars/8d3d0a2922f2e17610cb36e84af73584.svg
Yimin DU
user
yyy99
67d3a87df6ea55297c3cd078
Junchen Liu
admin_assigned
2025-03-14T11:07:42.920Z
663a9ae58cf658ffaf3d02e5
/avatars/aa35668dc088e675e794b9ceb935c56f.svg
Junchen Liu
user
JunchenLiu
67d3a87df6ea55297c3cd079
Lifu Tang
admin_assigned
2025-03-14T11:07:54.728Z
641d1c7045487810d137866c
/avatars/dc1140875c23213db4991714bdb1a3a7.svg
tlf
user
tanglifu
67d3a87df6ea55297c3cd07a
Xiaowei Lv
67d3a87df6ea55297c3cd07b
Haosheng Zou
admin_assigned
2025-03-14T11:06:40.468Z
65685c676c1ae87d4ee8dcfb
https://cdn-avatars.huggingface.co/v1/production/uploads/65685c676c1ae87d4ee8dcfb/YjUejWm43ODaiA4gAxvp1.jpeg
Haosheng Zou
user
zhs12
67d3a87df6ea55297c3cd07c
Yongchao Deng
67d3a87df6ea55297c3cd07d
Shousheng Jia
admin_assigned
2025-03-14T11:06:25.241Z
660a339940e346fba5b66f1d
/avatars/ae8e29f0ab259257af3e888553ef5946.svg
jiashousheng
user
shousheng
67d3a87df6ea55297c3cd07e
Xiangzheng Zhang
Light-R1: Curriculum SFT, DPO and RL for Long COT from Scratch and
  Beyond
This paper presents our work on the Light-R1 series, with models, data, and
code all released.
  We first focus on training long COT models from scratch, specifically
starting from models initially lacking long COT capabilities. Using a
curriculum training recipe consisting of two-stage SFT and semi-on-policy DPO,
we train our model Light-R1-32B from Qwen2.5-32B-Instruct, resulting in
superior math performance compared to DeepSeek-R1-Distill-Qwen-32B. Despite
being trained exclusively on math data, Light-R1-32B shows strong
generalization across other domains. In the subsequent phase of this work, we
highlight the significant benefit of the 3k dataset constructed for the second
SFT stage on enhancing other models. By fine-tuning DeepSeek-R1-Distilled
models using this dataset, we obtain new SOTA models in 7B and 14B, while the
32B model, Light-R1-32B-DS performed comparably to QwQ-32B and DeepSeek-R1.
  Furthermore, we extend our work by applying reinforcement learning,
specifically GRPO, on long-COT models to further improve reasoning performance.
We successfully train our final Light-R1-14B-DS with RL, achieving SOTA
performance among 14B parameter models in math. With AIME24 & 25 scores of 74.0
and 60.2 respectively, Light-R1-14B-DS surpasses even many 32B models and
DeepSeek-R1-Distill-Llama-70B. Its RL training also exhibits well expected
behavior, showing simultaneous increase in response length and reward score.
  The Light-R1 series of work validates training long-COT models from scratch,
showcases the art in SFT data and releases SOTA models from RL.
The Light-R1 series develops strong long-chain-of-thought models through curriculum training, semi-on-policy DPO, and reinforcement learning, achieving state-of-the-art performance in math and general domains.
curriculum training semi-on-policy DPO long COT models reinforcement learning GRPO