rStar2-Agent: Agentic Reasoning Technical Report
We introduce rStar2-Agent, a 14B math reasoning model trained with agentic
reinforcement learning to achieve frontier-level performance. Beyond current
long CoT, the model demonstrates advanced cognitive behaviors, such as thinking
carefully before using Python coding tools and reflecting on code execution
feedback to autonomously explore, verify, and refine intermediate steps in
complex problem-solving. This capability is enabled through three key
innovations that makes agentic RL effective at scale: (i) an efficient RL
infrastructure with a reliable Python code environment that supports
high-throughput execution and mitigates the high rollout costs, enabling
training on limited GPU resources (64 MI300X GPUs); (ii) GRPO-RoC, an agentic
RL algorithm with a Resample-on-Correct rollout strategy that addresses the
inherent environment noises from coding tools, allowing the model to reason
more effectively in a code environment; (iii) An efficient agent training
recipe that starts with non-reasoning SFT and progresses through multi-RL
stages, yielding advanced cognitive abilities with minimal compute cost. To
this end, rStar2-Agent boosts a pre-trained 14B model to state of the art in
only 510 RL steps within one week, achieving average pass@1 scores of 80.6% on
AIME24 and 69.8% on AIME25, surpassing DeepSeek-R1 (671B) with significantly
shorter responses. Beyond mathematics, rStar2-Agent-14B also demonstrates
strong generalization to alignment, scientific reasoning, and agentic tool-use
tasks. Code and training recipes are available at
https://github.com/microsoft/rStar.
68b10494b19c5400014848cb
Ning Shang
68b10494b19c5400014848cc
Yifei Liu
68b10494b19c5400014848cd
Yi Zhu
68b10494b19c5400014848ce
Li Lyna Zhang
claimed_verified
2025-08-29T11:09:40.680Z
62b0009c72043b05d29492b2
https://cdn-avatars.huggingface.co/v1/production/uploads/62b0009c72043b05d29492b2/NqRkX2YLhlfOLvYysa7dD.png
Li Lyna Zhang
user
lynazhang
68b10494b19c5400014848cf
Weijiang Xu
68b10494b19c5400014848d0
Xinyu Guan
68b10494b19c5400014848d1
Buze Zhang
68b10494b19c5400014848d2
Bingcheng Dong
68b10494b19c5400014848d3
Xudong Zhou
68b10494b19c5400014848d4
Bowen Zhang
68b10494b19c5400014848d5
Ying Xin
68b10494b19c5400014848d6
Ziming Miao
68b10494b19c5400014848d7
Scarlett Li
68b10494b19c5400014848d8
Fan Yang
68b10494b19c5400014848d9
Mao Yang
rStar2-Agent: Agentic Reasoning Technical Report
We introduce rStar2-Agent, a 14B math reasoning model trained with agentic
reinforcement learning to achieve frontier-level performance. Beyond current
long CoT, the model demonstrates advanced cognitive behaviors, such as thinking
carefully before using Python coding tools and reflecting on code execution
feedback to autonomously explore, verify, and refine intermediate steps in
complex problem-solving. This capability is enabled through three key
innovations that makes agentic RL effective at scale: (i) an efficient RL
infrastructure with a reliable Python code environment that supports
high-throughput execution and mitigates the high rollout costs, enabling
training on limited GPU resources (64 MI300X GPUs); (ii) GRPO-RoC, an agentic
RL algorithm with a Resample-on-Correct rollout strategy that addresses the
inherent environment noises from coding tools, allowing the model to reason
more effectively in a code environment; (iii) An efficient agent training
recipe that starts with non-reasoning SFT and progresses through multi-RL
stages, yielding advanced cognitive abilities with minimal compute cost. To
this end, rStar2-Agent boosts a pre-trained 14B model to state of the art in
only 510 RL steps within one week, achieving average pass@1 scores of 80.6% on
AIME24 and 69.8% on AIME25, surpassing DeepSeek-R1 (671B) with significantly
shorter responses. Beyond mathematics, rStar2-Agent-14B also demonstrates
strong generalization to alignment, scientific reasoning, and agentic tool-use
tasks. Code and training recipes are available at
https://github.com/microsoft/rStar.
rStar2-Agent, a 14B math reasoning model trained with agentic reinforcement learning, achieves state-of-the-art performance by efficiently handling complex problem-solving with advanced cognitive behaviors and minimal computational resources.
agentic reinforcement learning CoT Python coding tools GRPO-RoC Resample-on-Correct SFT multi-RL AIME24 AIME25 DeepSeek-R1 alignment scientific reasoning agentic tool-use