AceMath: Advancing Frontier Math Reasoning with Post-Training and Reward
  Modeling
In this paper, we introduce AceMath, a suite of frontier math models that
excel in solving complex math problems, along with highly effective reward
models capable of evaluating generated solutions and reliably identifying the
correct ones. To develop the instruction-tuned math models, we propose a
supervised fine-tuning (SFT) process that first achieves competitive
performance across general domains, followed by targeted fine-tuning for the
math domain using a carefully curated set of prompts and synthetically
generated responses. The resulting model, AceMath-72B-Instruct greatly
outperforms Qwen2.5-Math-72B-Instruct, GPT-4o and Claude-3.5 Sonnet. To develop
math-specialized reward model, we first construct AceMath-RewardBench, a
comprehensive and robust benchmark for evaluating math reward models across
diverse problems and difficulty levels. After that, we present a systematic
approach to build our math reward models. The resulting model, AceMath-72B-RM,
consistently outperforms state-of-the-art reward models. Furthermore, when
combining AceMath-72B-Instruct with AceMath-72B-RM, we achieve the highest
average rm@8 score across the math reasoning benchmarks. We will release model
weights, training data, and evaluation benchmarks at:
https://research.nvidia.com/labs/adlr/acemath
6764e3e30afbb34519fd2018
Zihan Liu
extracted_confirmed
2025-06-16T22:59:14.705Z
65f33b1c9f7970ccc0234cbf
/avatars/99fbab303912e3674663251c04279907.svg
Zihan Liu
user
zihanliu
6764e3e30afbb34519fd2019
Yang Chen
claimed_verified
2024-12-20T08:32:40.034Z
62bc9d90e81dfd65cced9316
/avatars/05df14cd1fdbc7d6a80d2960a05a94f0.svg
Yang Chen
user
ychenNLP
6764e3e30afbb34519fd201a
Mohammad Shoeybi
admin_assigned
2024-12-20T09:27:29.660Z
6641544c695975af2cbd0da6
/avatars/0ad3c18dcba585259b064fe9b00a07ce.svg
Mohammad Shoeybi
user
shoeybi
6764e3e30afbb34519fd201b
Bryan Catanzaro
admin_assigned
2024-12-20T09:27:23.912Z
6311021788942700629e6247
/avatars/e7adc1632b76e80e7e4a590033d1c20a.svg
Bryan Catanzaro
user
ctnzr
6764e3e30afbb34519fd201c
Wei Ping
extracted_pending
2024-12-20T03:26:28.621Z
663ee43bfeeb49803537da98
/avatars/17c3e9c435cc36fb04b4589e6176a243.svg
Wei Ping
user
wping
AceMath: Advancing Frontier Math Reasoning with Post-Training and Reward
  Modeling
In this paper, we introduce AceMath, a suite of frontier math models that
excel in solving complex math problems, along with highly effective reward
models capable of evaluating generated solutions and reliably identifying the
correct ones. To develop the instruction-tuned math models, we propose a
supervised fine-tuning (SFT) process that first achieves competitive
performance across general domains, followed by targeted fine-tuning for the
math domain using a carefully curated set of prompts and synthetically
generated responses. The resulting model, AceMath-72B-Instruct greatly
outperforms Qwen2.5-Math-72B-Instruct, GPT-4o and Claude-3.5 Sonnet. To develop
math-specialized reward model, we first construct AceMath-RewardBench, a
comprehensive and robust benchmark for evaluating math reward models across
diverse problems and difficulty levels. After that, we present a systematic
approach to build our math reward models. The resulting model, AceMath-72B-RM,
consistently outperforms state-of-the-art reward models. Furthermore, when
combining AceMath-72B-Instruct with AceMath-72B-RM, we achieve the highest
average rm@8 score across the math reasoning benchmarks. We will release model
weights, training data, and evaluation benchmarks at:
https://research.nvidia.com/labs/adlr/acemath
supervised fine-tuning (SFT) instruction-tuned math models fine-tuning prompts synthetically generated responses vision-and-language models reward models AceMath-RewardBench benchmark math reasoning benchmarks ACEMath-72B-Instruct Qwen2.5-Math-72B-Instruct GPT-4o Claude-3.5 mathematical problem span highlighting feature extraction feature representation rm@8 score model evaluation