MM-PRM: Enhancing Multimodal Mathematical Reasoning with Scalable
  Step-Level Supervision
While Multimodal Large Language Models (MLLMs) have achieved impressive
progress in vision-language understanding, they still struggle with complex
multi-step reasoning, often producing logically inconsistent or partially
correct solutions. A key limitation lies in the lack of fine-grained
supervision over intermediate reasoning steps. To address this, we propose
MM-PRM, a process reward model trained within a fully automated, scalable
framework. We first build MM-Policy, a strong multimodal model trained on
diverse mathematical reasoning data. Then, we construct MM-K12, a curated
dataset of 10,000 multimodal math problems with verifiable answers, which
serves as seed data. Leveraging a Monte Carlo Tree Search (MCTS)-based
pipeline, we generate over 700k step-level annotations without human labeling.
The resulting PRM is used to score candidate reasoning paths in the Best-of-N
inference setup and achieves significant improvements across both in-domain
(MM-K12 test set) and out-of-domain (OlympiadBench, MathVista, etc.)
benchmarks. Further analysis confirms the effectiveness of soft labels, smaller
learning rates, and path diversity in optimizing PRM performance. MM-PRM
demonstrates that process supervision is a powerful tool for enhancing the
logical robustness of multimodal reasoning systems. We release all our codes
and data at https://github.com/ModalMinds/MM-PRM.
682bfa77444a7d5f589a8769
Lingxiao Du
admin_assigned
2025-05-20T08:27:27.006Z
666fe1a5b07525f0bde69c27
/avatars/bb98ab0b974c8fe011739baa8dadd91a.svg
Lingxiao Du
user
Cierra0506
682bfa77444a7d5f589a876a
Fanqing Meng
admin_assigned
2025-05-20T08:27:33.802Z
640b37b2bab5ca8fbe7df8f2
/avatars/c7bef45efad6a0d911a720e2236fcba5.svg
fanqing meng
user
FanqingM
682bfa77444a7d5f589a876b
Zongkai Liu
claimed_verified
2025-05-21T09:03:38.181Z
674d42a03a4b7e31a1707218
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/3DIez-RYnDMYe1U-m0qBZ.png
kkkai
user
Zkkkai
682bfa77444a7d5f589a876c
Zhixiang Zhou
admin_assigned
2025-05-20T08:28:03.869Z
674bfdf227f531cdc248bb5c
https://cdn-avatars.huggingface.co/v1/production/uploads/674bfdf227f531cdc248bb5c/xh4gw89sr8MzNzRdiTjFx.jpeg
Zhixiang Zhou
user
SuperposedWave
682bfa77444a7d5f589a876d
Ping Luo
682bfa77444a7d5f589a876e
Qiaosheng Zhang
admin_assigned
2025-05-20T08:28:11.669Z
63cf4ecdc1dedf59c8f8362e
/avatars/cede885854d6a1551860080d55c87568.svg
Qiaosheng ZHANG
user
Domingo12
682bfa77444a7d5f589a876f
Wenqi Shao
admin_assigned
2025-05-20T08:28:17.437Z
64b3fd42eec33e27dcc4c941
/avatars/5aa1a99468fa61d4b8b0e80b592c4e55.svg
Wenqi Shao
user
wqshao126
MM-PRM: Enhancing Multimodal Mathematical Reasoning with Scalable
  Step-Level Supervision
While Multimodal Large Language Models (MLLMs) have achieved impressive
progress in vision-language understanding, they still struggle with complex
multi-step reasoning, often producing logically inconsistent or partially
correct solutions. A key limitation lies in the lack of fine-grained
supervision over intermediate reasoning steps. To address this, we propose
MM-PRM, a process reward model trained within a fully automated, scalable
framework. We first build MM-Policy, a strong multimodal model trained on
diverse mathematical reasoning data. Then, we construct MM-K12, a curated
dataset of 10,000 multimodal math problems with verifiable answers, which
serves as seed data. Leveraging a Monte Carlo Tree Search (MCTS)-based
pipeline, we generate over 700k step-level annotations without human labeling.
The resulting PRM is used to score candidate reasoning paths in the Best-of-N
inference setup and achieves significant improvements across both in-domain
(MM-K12 test set) and out-of-domain (OlympiadBench, MathVista, etc.)
benchmarks. Further analysis confirms the effectiveness of soft labels, smaller
learning rates, and path diversity in optimizing PRM performance. MM-PRM
demonstrates that process supervision is a powerful tool for enhancing the
logical robustness of multimodal reasoning systems. We release all our codes
and data at https://github.com/ModalMinds/MM-PRM.
MM-PRM, a process reward model with step-level annotations, enhances logical reasoning in multimodal language models by using automated supervision and achieves improved performance on various benchmarks.
Multimodal Large Language Models (MLLMs) MM-Policy MM-K12 Monte Carlo Tree Search (MCTS) process reward model Best-of-N inference logical robustness multimodal reasoning systems soft labels path diversity