Absolute Zero: Reinforced Self-play Reasoning with Zero Data
Reinforcement learning with verifiable rewards (RLVR) has shown promise in
enhancing the reasoning capabilities of large language models by learning
directly from outcome-based rewards. Recent RLVR works that operate under the
zero setting avoid supervision in labeling the reasoning process, but still
depend on manually curated collections of questions and answers for training.
The scarcity of high-quality, human-produced examples raises concerns about the
long-term scalability of relying on human supervision, a challenge already
evident in the domain of language model pretraining. Furthermore, in a
hypothetical future where AI surpasses human intelligence, tasks provided by
humans may offer limited learning potential for a superintelligent system. To
address these concerns, we propose a new RLVR paradigm called Absolute Zero, in
which a single model learns to propose tasks that maximize its own learning
progress and improves reasoning by solving them, without relying on any
external data. Under this paradigm, we introduce the Absolute Zero Reasoner
(AZR), a system that self-evolves its training curriculum and reasoning ability
by using a code executor to both validate proposed code reasoning tasks and
verify answers, serving as an unified source of verifiable reward to guide
open-ended yet grounded learning. Despite being trained entirely without
external data, AZR achieves overall SOTA performance on coding and mathematical
reasoning tasks, outperforming existing zero-setting models that rely on tens
of thousands of in-domain human-curated examples. Furthermore, we demonstrate
that AZR can be effectively applied across different model scales and is
compatible with various model classes.
681ab9b8f43603c60cab87a3
Andrew Zhao
claimed_verified
2025-05-14T07:36:10.447Z
630482fbce6b12280b18971d
/avatars/b07f31fd970d736bdf574d56da7a5634.svg
Andrew Zhao
user
andrewzh
681ab9b8f43603c60cab87a4
Yiran Wu
claimed_verified
2025-05-07T08:09:25.939Z
647eb24118274bce0308b2b8
/avatars/463e49a89b61164ccfad85ced10658b2.svg
Yiran Wu
user
kevinwyr
681ab9b8f43603c60cab87a5
Yang Yue
claimed_verified
2025-05-07T08:09:23.436Z
649d475111592b1a765ac1a3
https://cdn-avatars.huggingface.co/v1/production/uploads/649d475111592b1a765ac1a3/rjORJjErJq-mthghan08U.jpeg
Yang Yue
user
Yang130
681ab9b8f43603c60cab87a6
Tong Wu
681ab9b8f43603c60cab87a7
Quentin Xu
681ab9b8f43603c60cab87a8
Yang Yue
681ab9b8f43603c60cab87a9
Matthieu Lin
681ab9b8f43603c60cab87aa
Shenzhi Wang
claimed_verified
2025-05-07T08:09:28.623Z
6486dde1f74857df3f1a5828
https://cdn-avatars.huggingface.co/v1/production/uploads/6486dde1f74857df3f1a5828/FgE80CpalBO5qqArdfwxA.jpeg
Shenzhi Wang
user
shenzhi-wang
681ab9b8f43603c60cab87ab
Qingyun Wu
claimed_verified
2025-05-16T07:12:53.632Z
647fa485d0cd8be13e662973
https://cdn-avatars.huggingface.co/v1/production/uploads/647fa485d0cd8be13e662973/_QLgqLWhWir8_bkDCdTx6.png
Qingyun Wu
user
qingyun-wu
681ab9b8f43603c60cab87ac
Zilong Zheng
extracted_pending
2025-05-07T01:39:06.052Z
63a95a6a7930fa8c7dd63d4e
/avatars/d9d0420f7ddfe2f3a7e029fb05f1c89f.svg
Zilong Zheng
user
zlzheng
681ab9b8f43603c60cab87ad
Gao Huang
Absolute Zero: Reinforced Self-play Reasoning with Zero Data
Reinforcement learning with verifiable rewards (RLVR) has shown promise in
enhancing the reasoning capabilities of large language models by learning
directly from outcome-based rewards. Recent RLVR works that operate under the
zero setting avoid supervision in labeling the reasoning process, but still
depend on manually curated collections of questions and answers for training.
The scarcity of high-quality, human-produced examples raises concerns about the
long-term scalability of relying on human supervision, a challenge already
evident in the domain of language model pretraining. Furthermore, in a
hypothetical future where AI surpasses human intelligence, tasks provided by
humans may offer limited learning potential for a superintelligent system. To
address these concerns, we propose a new RLVR paradigm called Absolute Zero, in
which a single model learns to propose tasks that maximize its own learning
progress and improves reasoning by solving them, without relying on any
external data. Under this paradigm, we introduce the Absolute Zero Reasoner
(AZR), a system that self-evolves its training curriculum and reasoning ability
by using a code executor to both validate proposed code reasoning tasks and
verify answers, serving as an unified source of verifiable reward to guide
open-ended yet grounded learning. Despite being trained entirely without
external data, AZR achieves overall SOTA performance on coding and mathematical
reasoning tasks, outperforming existing zero-setting models that rely on tens
of thousands of in-domain human-curated examples. Furthermore, we demonstrate
that AZR can be effectively applied across different model scales and is
compatible with various model classes.
Absolute Zero Reasoner (AZR) achieves state-of-the-art performance on coding and mathematical reasoning tasks through self-generated, verified learning without external data.
reinforcement learning with verifiable rewards zero setting self-evolving training curriculum code executor verifiable reward open-ended learning grounded learning