MATH-Beyond: A Benchmark for RL to Expand Beyond the Base Model
With the advent of DeepSeek-R1, a new wave of reinforcement learning (RL)
methods has emerged that seem to unlock stronger mathematical reasoning.
However, a closer look at the open-source ecosystem reveals a critical
limitation: with sufficiently many draws (e.g., pass@1024), many
existing base models already solve nearly all questions on widely used math
benchmarks such as MATH-500 and AIME 2024. This suggests that the RL
fine-tuning methods prevalent in the LLM reasoning literature largely sharpen
existing solution modes rather than discovering entirely new ones. Such
sharpening stands in contrast to the broader promise of RL: to foster
exploration and to acquire new skills. To move beyond this plateau, we
introduce MATH-Beyond (MATH-B), a benchmark deliberately constructed to defeat
common open-source models of up to 8B parameters even under large sampling
budgets. Improving performance on our benchmark via RL requires methods that
learn to reason in ways that go beyond base model capabilities in repeated
sampling. Since the problems are drawn from subsets of DAPO-Math-17K and
DeepScaleR datasets, they remain topically equivalent to standard high-school
math. Validating our premise, RL fine-tuned models such as
Nemotron-Research-Reasoning-Qwen-1.5B and DeepScaleR-1.5B-Preview perform
poorly on MATH-B at pass@1024, showing how existing approaches fall
short on tackling harder instances. We hope MATH-B will catalyze
exploration-driven RL approaches that elicit deeper reasoning capabilities. We
release MATH-B at https://huggingface.co/datasets/brendel-group/MATH-Beyond.
68edca69de1fee572713a921
Prasanna Mayilvahanan
claimed_verified
2025-10-14T07:29:53.814Z
6698325e08250aac9f076f43
/avatars/f4a17acaf62ecf20e9ef54b166a891a9.svg
Prasanna Mayilvahanan
user
prasannamayil
68edca69de1fee572713a922
Ricardo Dominguez-Olmedo
68edca69de1fee572713a923
Thaddäus Wiedemer
68edca69de1fee572713a924
Wieland Brendel
MATH-Beyond: A Benchmark for RL to Expand Beyond the Base Model
With the advent of DeepSeek-R1, a new wave of reinforcement learning (RL)
methods has emerged that seem to unlock stronger mathematical reasoning.
However, a closer look at the open-source ecosystem reveals a critical
limitation: with sufficiently many draws (e.g., pass@1024), many
existing base models already solve nearly all questions on widely used math
benchmarks such as MATH-500 and AIME 2024. This suggests that the RL
fine-tuning methods prevalent in the LLM reasoning literature largely sharpen
existing solution modes rather than discovering entirely new ones. Such
sharpening stands in contrast to the broader promise of RL: to foster
exploration and to acquire new skills. To move beyond this plateau, we
introduce MATH-Beyond (MATH-B), a benchmark deliberately constructed to defeat
common open-source models of up to 8B parameters even under large sampling
budgets. Improving performance on our benchmark via RL requires methods that
learn to reason in ways that go beyond base model capabilities in repeated
sampling. Since the problems are drawn from subsets of DAPO-Math-17K and
DeepScaleR datasets, they remain topically equivalent to standard high-school
math. Validating our premise, RL fine-tuned models such as
Nemotron-Research-Reasoning-Qwen-1.5B and DeepScaleR-1.5B-Preview perform
poorly on MATH-B at pass@1024, showing how existing approaches fall
short on tackling harder instances. We hope MATH-B will catalyze
exploration-driven RL approaches that elicit deeper reasoning capabilities. We
release MATH-B at https://huggingface.co/datasets/brendel-group/MATH-Beyond.
MATH-Beyond is a benchmark designed to challenge existing reinforcement learning methods by requiring deeper reasoning capabilities beyond current model capabilities.
reinforcement learning RL fine-tuning MATH-500 AIME 2024 MATH-Beyond MATH-B DAPO-Math-17K DeepScaleR Nemotron-Research-Reasoning-Qwen-1.5B DeepScaleR-1.5B-Preview