Reinforcement Pre-Training
In this work, we introduce Reinforcement Pre-Training (RPT) as a new scaling
paradigm for large language models and reinforcement learning (RL).
Specifically, we reframe next-token prediction as a reasoning task trained
using RL, where it receives verifiable rewards for correctly predicting the
next token for a given context. RPT offers a scalable method to leverage vast
amounts of text data for general-purpose RL, rather than relying on
domain-specific annotated answers. By incentivizing the capability of
next-token reasoning, RPT significantly improves the language modeling accuracy
of predicting the next tokens. Moreover, RPT provides a strong pre-trained
foundation for further reinforcement fine-tuning. The scaling curves show that
increased training compute consistently improves the next-token prediction
accuracy. The results position RPT as an effective and promising scaling
paradigm to advance language model pre-training.
684794553ec10bdd8ab4de1a
Qingxiu Dong
684794553ec10bdd8ab4de1b
Li Dong
claimed_verified
2025-06-10T08:44:23.723Z
5df85abada6d0311fd3d5408
/avatars/2331cf703c1b5d3a62e2050b1a6eb108.svg
Li Dong
user
unilm
684794553ec10bdd8ab4de1c
Yao Tang
claimed_verified
2025-06-10T08:44:20.414Z
667119d6578448466d9531a6
/avatars/72c31909a5584b1306b6404b94a22b2a.svg
Yao Tang
user
YaoTang23
684794553ec10bdd8ab4de1d
Tianzhu Ye
claimed_verified
2025-06-11T08:36:07.338Z
64ac0605091ff88865352b44
/avatars/fac6c81110646251cfa694425927e74f.svg
ytz
user
ytz20
684794553ec10bdd8ab4de1e
Yutao Sun
684794553ec10bdd8ab4de1f
Zhifang Sui
684794553ec10bdd8ab4de20
Furu Wei
extracted_pending
2025-06-10T02:11:34.050Z
67ecd6178647cfa1775f75ed
/avatars/98882cc58dc0a5de94df765d523d92c9.svg
FW
user
frontierai
Reinforcement Pre-Training
In this work, we introduce Reinforcement Pre-Training (RPT) as a new scaling
paradigm for large language models and reinforcement learning (RL).
Specifically, we reframe next-token prediction as a reasoning task trained
using RL, where it receives verifiable rewards for correctly predicting the
next token for a given context. RPT offers a scalable method to leverage vast
amounts of text data for general-purpose RL, rather than relying on
domain-specific annotated answers. By incentivizing the capability of
next-token reasoning, RPT significantly improves the language modeling accuracy
of predicting the next tokens. Moreover, RPT provides a strong pre-trained
foundation for further reinforcement fine-tuning. The scaling curves show that
increased training compute consistently improves the next-token prediction
accuracy. The results position RPT as an effective and promising scaling
paradigm to advance language model pre-training.
Reinforcement Pre-Training (RPT) improves language model accuracy through reinforcement learning and offers a scalable method for leveraging text data for general-purpose RL.
Reinforcement Pre-Training (RPT) next-token prediction reasoning task reinforcement learning (RL) verifiable rewards language modeling accuracy reinforcement fine-tuning scaling curves