R1-ShareVL: Incentivizing Reasoning Capability of Multimodal Large
  Language Models via Share-GRPO
In this work, we aim to incentivize the reasoning ability of Multimodal Large
Language Models (MLLMs) via reinforcement learning (RL) and develop an
effective approach that mitigates the sparse reward and advantage vanishing
issues during RL. To this end, we propose Share-GRPO, a novel RL approach that
tackle these issues by exploring and sharing diverse reasoning trajectories
over expanded question space. Specifically, Share-GRPO first expands the
question space for a given question via data transformation techniques, and
then encourages MLLM to effectively explore diverse reasoning trajectories over
the expanded question space and shares the discovered reasoning trajectories
across the expanded questions during RL. In addition, Share-GRPO also shares
reward information during advantage computation, which estimates solution
advantages hierarchically across and within question variants, allowing more
accurate estimation of relative advantages and improving the stability of
policy training. Extensive evaluations over six widely-used reasoning
benchmarks showcase the superior performance of our method. Code will be
available at https://github.com/HJYao00/R1-ShareVL.
6836164664810fd39f82cf6e
Huanjin Yao
admin_assigned
2025-05-28T12:36:18.475Z
6590e03454f8826173ed5ee6
/avatars/f5e59d3e58c28a99f2ff39267ca51cdb.svg
Huanjin Yao
user
HuanjinYao
6836164664810fd39f82cf6f
Qixiang Yin
admin_assigned
2025-05-28T12:36:25.597Z
66841df7f0236757f437be00
/avatars/930d58451152bfd90a0e5549552790c0.svg
Qixiang Yin
user
Lucasqixiang
6836164664810fd39f82cf70
Jingyi Zhang
6836164664810fd39f82cf71
Min Yang
6836164664810fd39f82cf72
Yibo Wang
6836164664810fd39f82cf73
Wenhao Wu
6836164664810fd39f82cf74
Fei Su
6836164664810fd39f82cf75
Li Shen
6836164664810fd39f82cf76
Minghui Qiu
6836164664810fd39f82cf77
Dacheng Tao
6836164664810fd39f82cf78
Jiaxing Huang
admin_assigned
2025-05-28T12:37:33.188Z
67f3a9ec327c0599a0e7c2f0
/avatars/41674aad656c49487cb18fdd2a44c07c.svg
Huang jiaxing
user
WisteriaWilson
R1-ShareVL: Incentivizing Reasoning Capability of Multimodal Large
  Language Models via Share-GRPO
In this work, we aim to incentivize the reasoning ability of Multimodal Large
Language Models (MLLMs) via reinforcement learning (RL) and develop an
effective approach that mitigates the sparse reward and advantage vanishing
issues during RL. To this end, we propose Share-GRPO, a novel RL approach that
tackle these issues by exploring and sharing diverse reasoning trajectories
over expanded question space. Specifically, Share-GRPO first expands the
question space for a given question via data transformation techniques, and
then encourages MLLM to effectively explore diverse reasoning trajectories over
the expanded question space and shares the discovered reasoning trajectories
across the expanded questions during RL. In addition, Share-GRPO also shares
reward information during advantage computation, which estimates solution
advantages hierarchically across and within question variants, allowing more
accurate estimation of relative advantages and improving the stability of
policy training. Extensive evaluations over six widely-used reasoning
benchmarks showcase the superior performance of our method. Code will be
available at https://github.com/HJYao00/R1-ShareVL.
Share-GRPO, a novel reinforcement learning approach, enhances Multimodal Large Language Models by expanding the question space, sharing diverse reasoning trajectories, and hierarchical advantage computation.
Multimodal Large Language Models reinforcement learning sparse reward advantage vanishing Share-GRPO data transformation techniques reasoning trajectories question space hierarchical advantage computation