MMR1: Enhancing Multimodal Reasoning with Variance-Aware Sampling and
  Open Resources
Large multimodal reasoning models have achieved rapid progress, but their
advancement is constrained by two major limitations: the absence of open,
large-scale, high-quality long chain-of-thought (CoT) data, and the instability
of reinforcement learning (RL) algorithms in post-training. Group Relative
Policy Optimization (GRPO), the standard framework for RL fine-tuning, is prone
to gradient vanishing when reward variance is low, which weakens optimization
signals and impairs convergence. This work makes three contributions: (1) We
propose Variance-Aware Sampling (VAS), a data selection strategy guided by
Variance Promotion Score (VPS) that combines outcome variance and trajectory
diversity to promote reward variance and stabilize policy optimization. (2) We
release large-scale, carefully curated resources containing ~1.6M long CoT
cold-start data and ~15k RL QA pairs, designed to ensure quality, difficulty,
and diversity, along with a fully reproducible end-to-end training codebase.
(3) We open-source a family of multimodal reasoning models in multiple scales,
establishing standardized baselines for the community. Experiments across
mathematical reasoning benchmarks demonstrate the effectiveness of both the
curated data and the proposed VAS. Comprehensive ablation studies and analyses
provide further insight into the contributions of each component. In addition,
we theoretically establish that reward variance lower-bounds the expected
policy gradient magnitude, with VAS serving as a practical mechanism to realize
this guarantee. Our code, data, and checkpoints are available at
https://github.com/LengSicong/MMR1.
68d5f4108ccd91bdd39ffd93
Sicong Leng
68d5f4108ccd91bdd39ffd94
Jing Wang
68d5f4108ccd91bdd39ffd95
Jiaxi Li
claimed_verified
2025-09-26T12:19:42.255Z
64c90edf0986bd6fa23f3c2d
/avatars/de778ea9ef17397f3839fc4d4bbf6c06.svg
Jiaxi
user
jxjessieli
68d5f4108ccd91bdd39ffd96
Hao Zhang
claimed_verified
2025-09-26T12:19:48.387Z
64b7cd74ff6d81ae297feded
/avatars/880fbc96cc093f5e901ce84f32a1d21d.svg
ZHANG HAO
user
26hzhang
68d5f4108ccd91bdd39ffd97
Zhiqiang Hu
68d5f4108ccd91bdd39ffd98
Boqiang Zhang
68d5f4108ccd91bdd39ffd99
Yuming Jiang
admin_assigned
2025-09-26T16:07:04.855Z
629c95b7a5d6f5fe10e6ed45
https://cdn-avatars.huggingface.co/v1/production/uploads/629c95b7a5d6f5fe10e6ed45/Sy0Ype5snsRookID-gsSm.jpeg
Yuming Jiang
user
yumingj
68d5f4108ccd91bdd39ffd9a
Hang Zhang
68d5f4108ccd91bdd39ffd9b
Xin Li
68d5f4108ccd91bdd39ffd9c
Lidong Bing
admin_assigned
2025-09-26T16:07:55.992Z
6454685a548f22be598414c4
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/eMjMWKJ-AouF7eY1-RzGF.jpeg
Lidong Bing
user
LidongBing
68d5f4108ccd91bdd39ffd9d
Deli Zhao
68d5f4108ccd91bdd39ffd9e
Wei Lu
68d5f4108ccd91bdd39ffd9f
Yu Rong
claimed_verified
2025-09-26T12:19:37.968Z
642eecbf9b2484d7d8526781
https://cdn-avatars.huggingface.co/v1/production/uploads/642eecbf9b2484d7d8526781/4IvGbd66s49Wx5pZyZGHA.png
Yu Rong
user
Swrooy
68d5f4108ccd91bdd39ffda0
Aixin Sun
admin_assigned
2025-09-26T16:07:32.383Z
664aab898fa42b4fe70ebf52
/avatars/a38455fd17bbc74ce3111f2c3da9aa59.svg
Aixin Sun
user
aixinsun
68d5f4108ccd91bdd39ffda1
Shijian Lu
MMR1: Enhancing Multimodal Reasoning with Variance-Aware Sampling and
  Open Resources
Large multimodal reasoning models have achieved rapid progress, but their
advancement is constrained by two major limitations: the absence of open,
large-scale, high-quality long chain-of-thought (CoT) data, and the instability
of reinforcement learning (RL) algorithms in post-training. Group Relative
Policy Optimization (GRPO), the standard framework for RL fine-tuning, is prone
to gradient vanishing when reward variance is low, which weakens optimization
signals and impairs convergence. This work makes three contributions: (1) We
propose Variance-Aware Sampling (VAS), a data selection strategy guided by
Variance Promotion Score (VPS) that combines outcome variance and trajectory
diversity to promote reward variance and stabilize policy optimization. (2) We
release large-scale, carefully curated resources containing ~1.6M long CoT
cold-start data and ~15k RL QA pairs, designed to ensure quality, difficulty,
and diversity, along with a fully reproducible end-to-end training codebase.
(3) We open-source a family of multimodal reasoning models in multiple scales,
establishing standardized baselines for the community. Experiments across
mathematical reasoning benchmarks demonstrate the effectiveness of both the
curated data and the proposed VAS. Comprehensive ablation studies and analyses
provide further insight into the contributions of each component. In addition,
we theoretically establish that reward variance lower-bounds the expected
policy gradient magnitude, with VAS serving as a practical mechanism to realize
this guarantee. Our code, data, and checkpoints are available at
https://github.com/LengSicong/MMR1.
Variance-Aware Sampling and large-scale CoT data improve multimodal reasoning models by stabilizing RL fine-tuning and enhancing performance on benchmarks.
Variance-Aware Sampling Variance Promotion Score Group Relative Policy Optimization gradient vanishing reward variance policy optimization long chain-of-thought data multimodal reasoning models reinforcement learning fine-tuning mathematical reasoning benchmarks