Vision-R1: Incentivizing Reasoning Capability in Multimodal Large
  Language Models
DeepSeek-R1-Zero has successfully demonstrated the emergence of reasoning
capabilities in LLMs purely through Reinforcement Learning (RL). Inspired by
this breakthrough, we explore how RL can be utilized to enhance the reasoning
capability of MLLMs. However, direct training with RL struggles to activate
complex reasoning capabilities such as questioning and reflection in MLLMs, due
to the absence of substantial high-quality multimodal reasoning data. To
address this issue, we propose the reasoning MLLM, Vision-R1, to improve
multimodal reasoning capability. Specifically, we first construct a
high-quality multimodal CoT dataset without human annotations by leveraging an
existing MLLM and DeepSeek-R1 through modality bridging and data filtering to
obtain a 200K multimodal CoT dataset, Vision-R1-cold dataset. It serves as
cold-start initialization data for Vision-R1. To mitigate the optimization
challenges caused by overthinking after cold start, we propose Progressive
Thinking Suppression Training (PTST) strategy and employ Group Relative Policy
Optimization (GRPO) with the hard formatting result reward function to
gradually refine the model's ability to learn correct and complex reasoning
processes on a 10K multimodal math dataset. Comprehensive experiments show our
model achieves an average improvement of sim6% across various multimodal
math reasoning benchmarks. Vision-R1-7B achieves a 73.5% accuracy on the widely
used MathVista benchmark, which is only 0.4% lower than the leading reasoning
model, OpenAI O1. The datasets and code will be released in:
https://github.com/Osilly/Vision-R1 .
67cfb6495944a8e54f24cd9a
Wenxuan Huang
claimed_verified
2025-04-18T09:41:35.776Z
67dc162ec8c00778e8689f42
https://cdn-avatars.huggingface.co/v1/production/uploads/67dc162ec8c00778e8689f42/_y_tO6W3ONOkOWbumAFXA.png
Wenxuan Huang
user
Osilly
67cfb6495944a8e54f24cd9b
Bohan Jia
admin_assigned
2025-03-11T14:32:47.826Z
667507c64b1e6619167123b0
/avatars/f101cb9eb1acc0a2cb46f8320907b06f.svg
Bohan Jia
user
BohanJia
67cfb6495944a8e54f24cd9c
Zijie Zhai
admin_assigned
2025-03-11T14:32:54.315Z
6669981b9d56215237eaa25f
/avatars/ba8bd9b34637d1c52cc9ff17cab79da1.svg
Zijie Zhai
user
skyz1010
67cfb6495944a8e54f24cd9d
Shaosheng Cao
67cfb6495944a8e54f24cd9e
Zheyu Ye
admin_assigned
2025-03-11T14:33:09.475Z
61eeecc16f01506a837664ce
/avatars/61ef427943e2945f512e6e3c705991c7.svg
Zheyu Ye
user
zheyuye
67cfb6495944a8e54f24cd9f
Fei Zhao
67cfb6495944a8e54f24cda0
Yao Hu
67cfb6495944a8e54f24cda1
Shaohui Lin
admin_assigned
2025-03-11T14:33:21.819Z
6481ace6578646b5c2370b7e
/avatars/218ffacc705c1fe0e23ef1217f76b374.svg
linshaohui
user
kimlin123
Vision-R1: Incentivizing Reasoning Capability in Multimodal Large
  Language Models
DeepSeek-R1-Zero has successfully demonstrated the emergence of reasoning
capabilities in LLMs purely through Reinforcement Learning (RL). Inspired by
this breakthrough, we explore how RL can be utilized to enhance the reasoning
capability of MLLMs. However, direct training with RL struggles to activate
complex reasoning capabilities such as questioning and reflection in MLLMs, due
to the absence of substantial high-quality multimodal reasoning data. To
address this issue, we propose the reasoning MLLM, Vision-R1, to improve
multimodal reasoning capability. Specifically, we first construct a
high-quality multimodal CoT dataset without human annotations by leveraging an
existing MLLM and DeepSeek-R1 through modality bridging and data filtering to
obtain a 200K multimodal CoT dataset, Vision-R1-cold dataset. It serves as
cold-start initialization data for Vision-R1. To mitigate the optimization
challenges caused by overthinking after cold start, we propose Progressive
Thinking Suppression Training (PTST) strategy and employ Group Relative Policy
Optimization (GRPO) with the hard formatting result reward function to
gradually refine the model's ability to learn correct and complex reasoning
processes on a 10K multimodal math dataset. Comprehensive experiments show our
model achieves an average improvement of sim6% across various multimodal
math reasoning benchmarks. Vision-R1-7B achieves a 73.5% accuracy on the widely
used MathVista benchmark, which is only 0.4% lower than the leading reasoning
model, OpenAI O1. The datasets and code will be released in:
https://github.com/Osilly/Vision-R1 .
A reasoning MLLM called Vision-R1 is proposed to enhance multimodal reasoning capabilities using Reinforcement Learning and a constructed multimodal CoT dataset, achieving high performance on multimodal math reasoning benchmarks.
LLMs RL MLLMs multimodal reasoning Vision-R1 CoT dataset modality bridging data filtering Vision-R1-cold dataset Progressive Thinking Suppression Training (PTST) Group Relative Policy Optimization (GRPO) hard formatting result reward function MathVista benchmark