ProcessBench: Identifying Process Errors in Mathematical Reasoning
As language models regularly make mistakes when solving math problems,
automated identification of errors in the reasoning process becomes
increasingly significant for their scalable oversight. In this paper, we
introduce ProcessBench for measuring the ability to identify erroneous steps in
mathematical reasoning. It consists of 3,400 test cases, primarily focused on
competition- and Olympiad-level math problems. Each test case contains a
step-by-step solution with error location annotated by human experts. Models
are required to identify the earliest step that contains an error, or conclude
that all steps are correct. We conduct extensive evaluation on ProcessBench,
involving two types of models: process reward models (PRMs) and critic models,
where for the latter we prompt general language models to critique each
solution step by step. We draw two main observations: (1) Existing PRMs
typically fail to generalize to more challenging math problems beyond GSM8K and
MATH. They underperform both critic models (i.e., prompted general language
models) and our own trained PRM that is straightforwardly fine-tuned on the
PRM800K dataset. (2) The best open-source model, QwQ-32B-Preview, has
demonstrated the critique capability competitive with the proprietary model
GPT-4o, despite that it still lags behind the reasoning-specialized o1-mini. We
hope ProcessBench can foster future research in reasoning process assessment,
paving the way toward scalable oversight of language models.
6757e135887c48188a1f5015
Chujie Zheng
claimed_verified
2024-12-10T10:25:17.928Z
610b70452719facd4ea85e28
https://cdn-avatars.huggingface.co/v1/production/uploads/610b70452719facd4ea85e28/S7nMy7D0Rxq0VIVblhYDG.jpeg
Chujie Zheng
user
chujiezheng
6757e135887c48188a1f5016
Zhenru Zhang
admin_assigned
2024-12-10T16:13:54.636Z
652f13fa6504631c10f308ed
/avatars/440de01158c1d9152afbf33617fe27d1.svg
zhenrui
user
zhenrui
6757e135887c48188a1f5017
Beichen Zhang
6757e135887c48188a1f5018
Runji Lin
claimed_verified
2024-12-10T10:25:16.002Z
649a52e5de0fb7f3f499e583
/avatars/25f6106fa168ae57ad5cd8ef55c70d31.svg
Runji Lin
user
RunjiLin
6757e135887c48188a1f5019
Keming Lu
6757e135887c48188a1f501a
Bowen Yu
admin_assigned
2024-12-10T16:12:57.169Z
6438b43ab2ea24b52ebac2b9
/avatars/84133cd719a4b1e2f5c1a74178425f86.svg
Bowen Yu
user
bwy
6757e135887c48188a1f501b
Dayiheng Liu
admin_assigned
2024-12-10T16:12:41.742Z
6434d4989bd5a84b5dd0b0f5
https://cdn-avatars.huggingface.co/v1/production/uploads/6434d4989bd5a84b5dd0b0f5/0Elf9qbfG9Hkgypm9pTGm.jpeg
Dayiheng Liu
user
Losin94
6757e135887c48188a1f501c
Jingren Zhou
admin_assigned
2024-12-10T16:12:36.306Z
602f88f5e8149a962412a667
/avatars/b78f0e583df8e5d5e3365934fe5f4900.svg
Zhou
user
Jingren
6757e135887c48188a1f501d
Junyang Lin
admin_assigned
2024-12-10T16:12:26.676Z
620760a26e3b7210c2ff1943
https://cdn-avatars.huggingface.co/v1/production/uploads/620760a26e3b7210c2ff1943/VC-rKqimF6yxGESNVlPoR.jpeg
Junyang Lin
user
JustinLin610
ProcessBench: Identifying Process Errors in Mathematical Reasoning
As language models regularly make mistakes when solving math problems,
automated identification of errors in the reasoning process becomes
increasingly significant for their scalable oversight. In this paper, we
introduce ProcessBench for measuring the ability to identify erroneous steps in
mathematical reasoning. It consists of 3,400 test cases, primarily focused on
competition- and Olympiad-level math problems. Each test case contains a
step-by-step solution with error location annotated by human experts. Models
are required to identify the earliest step that contains an error, or conclude
that all steps are correct. We conduct extensive evaluation on ProcessBench,
involving two types of models: process reward models (PRMs) and critic models,
where for the latter we prompt general language models to critique each
solution step by step. We draw two main observations: (1) Existing PRMs
typically fail to generalize to more challenging math problems beyond GSM8K and
MATH. They underperform both critic models (i.e., prompted general language
models) and our own trained PRM that is straightforwardly fine-tuned on the
PRM800K dataset. (2) The best open-source model, QwQ-32B-Preview, has
demonstrated the critique capability competitive with the proprietary model
GPT-4o, despite that it still lags behind the reasoning-specialized o1-mini. We
hope ProcessBench can foster future research in reasoning process assessment,
paving the way toward scalable oversight of language models.
ProcessBench evaluates models' ability to identify errors in mathematical reasoning steps, showing that existing process reward models struggle with difficult problems and underperform compared to critic models and a fine-tuned PRM.
ProcessBench process reward models critic models error identification mathematical reasoning PRM800K dataset QwQ-32B-Preview GPT-4o reasoning-specialized o1-mini