DAPO: An Open-Source LLM Reinforcement Learning System at Scale
Inference scaling empowers LLMs with unprecedented reasoning ability, with
reinforcement learning as the core technique to elicit complex reasoning.
However, key technical details of state-of-the-art reasoning LLMs are concealed
(such as in OpenAI o1 blog and DeepSeek R1 technical report), thus the
community still struggles to reproduce their RL training results. We propose
the Decoupled Clip and Dynamic sAmpling
Policy Optimization (DAPO) algorithm, and
fully open-source a state-of-the-art large-scale RL system that achieves 50
points on AIME 2024 using Qwen2.5-32B base model. Unlike previous works that
withhold training details, we introduce four key techniques of our algorithm
that make large-scale LLM RL a success. In addition, we open-source our
training code, which is built on the verl framework, along with a carefully
curated and processed dataset. These components of our open-source system
enhance reproducibility and support future research in large-scale LLM RL.
67da2b54e5335651349e262c
Qiying Yu
admin_assigned
2025-03-19T13:19:18.079Z
6341375be5071b0c5cc946a3
/avatars/c26a32b54a3f7731854059d247b5c523.svg
Qiying Yu
user
qiying
67da2b54e5335651349e262d
Zheng Zhang
admin_assigned
2025-03-19T13:19:31.824Z
648e9244e6598457da62a180
/avatars/60329251f08e93260196ff67a842c188.svg
Zheng Zhang
user
zhengzhang
67da2b54e5335651349e262e
Ruofei Zhu
67da2b54e5335651349e262f
Yufeng Yuan
admin_assigned
2025-03-19T13:19:54.321Z
64b755e500bac1088cea02f1
/avatars/f4adf70175e84e945436d1f091ba338b.svg
Yufeng Yuan
user
yufeng10
67da2b54e5335651349e2630
Xiaochen Zuo
67da2b54e5335651349e2631
Yu Yue
67da2b54e5335651349e2632
Tiantian Fan
67da2b54e5335651349e2633
Gaohong Liu
admin_assigned
2025-03-19T13:20:22.957Z
61512f1b3e89795099b6fced
/avatars/8f19253df7343e9d95ad7d0b3d429c06.svg
Gaohong Liu
user
Cheimu
67da2b54e5335651349e2634
Lingjun Liu
admin_assigned
2025-03-19T13:20:29.992Z
6626274011772517e5469d48
/avatars/5f2606614740da04a7fb9d80fa628fb1.svg
Lingjun Liu
user
EEchollj
67da2b54e5335651349e2635
Xin Liu
67da2b54e5335651349e2636
Haibin Lin
admin_assigned
2025-03-19T13:20:36.744Z
65e809f67c1853c8c180baff
/avatars/e489eaebbbf86f8826fe0e9d5423be01.svg
haibin
user
haibinlin
67da2b54e5335651349e2637
Zhiqi Lin
67da2b54e5335651349e2638
Bole Ma
67da2b54e5335651349e2639
Guangming Sheng
67da2b54e5335651349e263a
Yuxuan Tong
admin_assigned
2025-03-19T13:21:29.044Z
6448e1fbe988635a3d6aa97d
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/eG4R9-3hgrimttP7ep3dN.jpeg
Shawn/Yuxuan Tong
user
tongyx361
67da2b54e5335651349e263b
Chi Zhang
67da2b54e5335651349e263c
Mofan Zhang
admin_assigned
2025-03-19T13:21:51.391Z
64f33235bbf2fd1b4816632b
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/sVN-M1Hj-aB4QqytdpUsn.jpeg
Mofan Zhang
user
NewMomo
67da2b54e5335651349e263d
Wang Zhang
67da2b54e5335651349e263e
Hang Zhu
67da2b54e5335651349e263f
Jinhua Zhu
admin_assigned
2025-03-19T13:22:23.684Z
650b0009bddcb62e458ebadf
/avatars/0431a38b9c9ca0e11ec505ce81183be8.svg
Jinhua Zhu
user
teslazhu
67da2b54e5335651349e2640
Jiaze Chen
67da2b54e5335651349e2641
Jiangjie Chen
claimed_verified
2025-05-27T07:56:06.181Z
606ed1884ffe81d1e03e81e5
https://cdn-avatars.huggingface.co/v1/production/uploads/1639375346654-606ed1884ffe81d1e03e81e5.png
Jiangjie Chen
user
jiangjiechen
67da2b54e5335651349e2642
Chengyi Wang
67da2b54e5335651349e2643
Hongli Yu
claimed_verified
2025-07-08T08:07:59.640Z
66c58a7e960ea4e66c632075
/avatars/527ad90f605e5aaf76298204dfa67d55.svg
hlyu
user
huiyeruzhou
67da2b54e5335651349e2644
Weinan Dai
67da2b54e5335651349e2645
Yuxuan Song
claimed_verified
2025-08-06T19:38:24.674Z
66275b53d138af2d2eeb9326
/avatars/1103f68b978238d5bce604294d467e00.svg
Yuxuan Song
user
yxsong
67da2b54e5335651349e2646
Xiangpeng Wei
67da2b54e5335651349e2647
Hao Zhou
67da2b54e5335651349e2648
Jingjing Liu
67da2b54e5335651349e2649
Wei-Ying Ma
67da2b54e5335651349e264a
Ya-Qin Zhang
admin_assigned
2025-03-19T13:22:33.159Z
656c1600665d15428a8bdc60
/avatars/dfdf5f8712c769896b4ab655c4ce32e4.svg
Ya-Qin Zhang
user
Ya-Qin
67da2b54e5335651349e264b
Lin Yan
67da2b54e5335651349e264c
Mu Qiao
67da2b54e5335651349e264d
Yonghui Wu
admin_assigned
2025-03-19T13:22:16.717Z
647a13ba5f3450e1ded4ca85
https://cdn-avatars.huggingface.co/v1/production/uploads/647a13ba5f3450e1ded4ca85/0ryiqG2gpFlffOFNq3jpj.jpeg
Yonghui Wu
user
yonghuiwu
67da2b54e5335651349e264e
Mingxuan Wang
DAPO: An Open-Source LLM Reinforcement Learning System at Scale
Inference scaling empowers LLMs with unprecedented reasoning ability, with
reinforcement learning as the core technique to elicit complex reasoning.
However, key technical details of state-of-the-art reasoning LLMs are concealed
(such as in OpenAI o1 blog and DeepSeek R1 technical report), thus the
community still struggles to reproduce their RL training results. We propose
the Decoupled Clip and Dynamic sAmpling
Policy Optimization (DAPO) algorithm, and
fully open-source a state-of-the-art large-scale RL system that achieves 50
points on AIME 2024 using Qwen2.5-32B base model. Unlike previous works that
withhold training details, we introduce four key techniques of our algorithm
that make large-scale LLM RL a success. In addition, we open-source our
training code, which is built on the verl framework, along with a carefully
curated and processed dataset. These components of our open-source system
enhance reproducibility and support future research in large-scale LLM RL.
The DAPO algorithm, which includes decoupled clip and dynamic sampling policy optimization, enables open-source, high-performance reinforcement learning training for large-scale language models, enhancing reproducibility in the field.
LLMs reinforcement learning decoupled clip dynamic sampling policy optimization DAPO large-scale RL open-source AIME 2024 Qwen2.5-32B verl framework