DeepMath-103K: A Large-Scale, Challenging, Decontaminated, and
  Verifiable Mathematical Dataset for Advancing Reasoning
The capacity for complex mathematical reasoning is a key benchmark for
artificial intelligence. While reinforcement learning (RL) applied to LLMs
shows promise, progress is significantly hindered by the lack of large-scale
training data that is sufficiently challenging, possesses verifiable answer
formats suitable for RL, and is free from contamination with evaluation
benchmarks. To address these limitations, we introduce DeepMath-103K, a new,
large-scale dataset comprising approximately 103K mathematical problems,
specifically designed to train advanced reasoning models via RL. DeepMath-103K
is curated through a rigorous pipeline involving source analysis, stringent
decontamination against numerous benchmarks, and filtering for high difficulty
(primarily Levels 5-9), significantly exceeding existing open resources in
challenge. Each problem includes a verifiable final answer, enabling rule-based
RL, and three distinct R1-generated solutions suitable for diverse training
paradigms like supervised fine-tuning or distillation. Spanning a wide range of
mathematical topics, DeepMath-103K promotes the development of generalizable
reasoning. We demonstrate that models trained on DeepMath-103K achieve
significant improvements on challenging mathematical benchmarks, validating its
effectiveness. We release DeepMath-103K publicly to facilitate community
progress in building more capable AI reasoning systems:
https://github.com/zwhe99/DeepMath.
67ff79b3d68757d92e9c168e
Zhiwei He
67ff79b3d68757d92e9c168f
Tian Liang
67ff79b3d68757d92e9c1690
Jiahao Xu
67ff79b3d68757d92e9c1691
Qiuzhi Liu
67ff79b3d68757d92e9c1692
Xingyu Chen
67ff79b3d68757d92e9c1693
Yue Wang
67ff79b3d68757d92e9c1694
Linfeng Song
67ff79b3d68757d92e9c1695
Dian Yu
67ff79b3d68757d92e9c1696
Zhenwen Liang
67ff79b3d68757d92e9c1697
Wenxuan Wang
67ff79b3d68757d92e9c1698
Zhuosheng Zhang
67ff79b3d68757d92e9c1699
Rui Wang
67ff79b3d68757d92e9c169a
Zhaopeng Tu
claimed_verified
2025-05-09T07:22:52.128Z
67485743561b1e6f9579389f
/avatars/8a4cc63bd7be388010bc329bb74582a1.svg
Zhaopeng Tu
user
zptu
67ff79b3d68757d92e9c169b
Haitao Mi
67ff79b3d68757d92e9c169c
Dong Yu
DeepMath-103K: A Large-Scale, Challenging, Decontaminated, and
  Verifiable Mathematical Dataset for Advancing Reasoning
The capacity for complex mathematical reasoning is a key benchmark for
artificial intelligence. While reinforcement learning (RL) applied to LLMs
shows promise, progress is significantly hindered by the lack of large-scale
training data that is sufficiently challenging, possesses verifiable answer
formats suitable for RL, and is free from contamination with evaluation
benchmarks. To address these limitations, we introduce DeepMath-103K, a new,
large-scale dataset comprising approximately 103K mathematical problems,
specifically designed to train advanced reasoning models via RL. DeepMath-103K
is curated through a rigorous pipeline involving source analysis, stringent
decontamination against numerous benchmarks, and filtering for high difficulty
(primarily Levels 5-9), significantly exceeding existing open resources in
challenge. Each problem includes a verifiable final answer, enabling rule-based
RL, and three distinct R1-generated solutions suitable for diverse training
paradigms like supervised fine-tuning or distillation. Spanning a wide range of
mathematical topics, DeepMath-103K promotes the development of generalizable
reasoning. We demonstrate that models trained on DeepMath-103K achieve
significant improvements on challenging mathematical benchmarks, validating its
effectiveness. We release DeepMath-103K publicly to facilitate community
progress in building more capable AI reasoning systems:
https://github.com/zwhe99/DeepMath.
A large-scale dataset, DeepMath-103K, comprising complex mathematical problems is introduced to enhance reinforcement learning-driven reasoning in models.
reinforcement learning R1-generated solutions supervised fine-tuning distillation generalizable reasoning