Skywork R1V2: Multimodal Hybrid Reinforcement Learning for Reasoning
We present Skywork R1V2, a next-generation multimodal reasoning model and a
major leap forward from its predecessor, Skywork R1V. At its core, R1V2
introduces a hybrid reinforcement learning paradigm that harmonizes
reward-model guidance with rule-based strategies, thereby addressing the
long-standing challenge of balancing sophisticated reasoning capabilities with
broad generalization. To further enhance training efficiency, we propose the
Selective Sample Buffer (SSB) mechanism, which effectively counters the
``Vanishing Advantages'' dilemma inherent in Group Relative Policy Optimization
(GRPO) by prioritizing high-value samples throughout the optimization process.
Notably, we observe that excessive reinforcement signals can induce visual
hallucinations--a phenomenon we systematically monitor and mitigate through
calibrated reward thresholds throughout the training process. Empirical results
affirm the exceptional capability of R1V2, with benchmark-leading performances
such as 62.6 on OlympiadBench, 79.0 on AIME2024, 63.6 on LiveCodeBench, and
74.0 on MMMU. These results underscore R1V2's superiority over existing
open-source models and demonstrate significant progress in closing the
performance gap with premier proprietary systems, including Gemini 2.5 and
OpenAI o4-mini. The Skywork R1V2 model weights have been publicly released to
promote openness and reproducibility
https://huggingface.co/Skywork/Skywork-R1V2-38B.
6809a4ac81a95c83f0c81c83
Chris
admin_assigned
2025-05-09T12:34:22.418Z
620f5a1c3f76c50e6458a9b6
https://cdn-avatars.huggingface.co/v1/production/uploads/620f5a1c3f76c50e6458a9b6/pXh_f5F0UvufxuUa-eS-v.jpeg
Peiyu Wang
user
OrlandoHugBot
6809a4ac81a95c83f0c81c84
Yichen Wei
6809a4ac81a95c83f0c81c85
Yi Peng
admin_assigned
2025-04-29T12:31:13.114Z
6321769fb8297a7899903f5c
/avatars/12c86bd9191cdfddf2cf693db3125be8.svg
PENG YI
user
pengyi
6809a4ac81a95c83f0c81c86
Xiaokun Wang
claimed_verified
2025-05-09T07:22:46.116Z
62be9b5aae56e75e4d689e7c
/avatars/6772bc09d6eeb4e86b1210481be91720.svg
wangxiaokun
user
shawn0wang
6809a4ac81a95c83f0c81c87
Weijie Qiu
admin_assigned
2025-04-29T12:30:41.500Z
660aab2c878289c5b34f9e97
https://cdn-avatars.huggingface.co/v1/production/uploads/660aab2c878289c5b34f9e97/yxx1-lR8x5o6KaEpZDXQq.jpeg
weijie qiu
user
qiuwj
6809a4ac81a95c83f0c81c88
Wei Shen
6809a4ac81a95c83f0c81c89
Tianyidan Xie
claimed_verified
2025-04-28T12:50:25.445Z
63fdb1aa27abbe6b3ce098f5
/avatars/c22e3a77ff84b3b87c16cff2469f6d3d.svg
xietian
user
sealical
6809a4ac81a95c83f0c81c8a
Jiangbo Pei
admin_assigned
2025-04-29T12:30:09.986Z
653df83feefdcdc9fd8a5b13
/avatars/f7f5bc67df7f8bf8b28e9bbd4fa6c397.svg
Jiangbo Pei
user
jiangbop
6809a4ac81a95c83f0c81c8b
Jianhao Zhang
admin_assigned
2025-04-29T12:29:46.829Z
64235adf23b98d5ee6e9ca85
https://cdn-avatars.huggingface.co/v1/production/uploads/64235adf23b98d5ee6e9ca85/m_7uwLaAkTXyt3qB2pPLA.png
Jianhao Zhang
user
Heerozh
6809a4ac81a95c83f0c81c8c
Yunzhuo Hao
admin_assigned
2025-04-29T12:29:40.419Z
653dd16277c2f09452ad37cd
/avatars/a95f9527722845a5414d86180c8e945d.svg
Yunzhuo Hao
user
luckychao
6809a4ac81a95c83f0c81c8d
Xuchen Song
claimed_verified
2025-04-25T08:35:17.241Z
6462b241b438438da3c25a5d
/avatars/606a67f1be639c9a5e36f293abd5f27a.svg
Xuchen Song
user
xuchensong
6809a4ac81a95c83f0c81c8e
Yang Liu
6809a4ac81a95c83f0c81c8f
Yahui Zhou
Skywork R1V2: Multimodal Hybrid Reinforcement Learning for Reasoning
We present Skywork R1V2, a next-generation multimodal reasoning model and a
major leap forward from its predecessor, Skywork R1V. At its core, R1V2
introduces a hybrid reinforcement learning paradigm that harmonizes
reward-model guidance with rule-based strategies, thereby addressing the
long-standing challenge of balancing sophisticated reasoning capabilities with
broad generalization. To further enhance training efficiency, we propose the
Selective Sample Buffer (SSB) mechanism, which effectively counters the
``Vanishing Advantages'' dilemma inherent in Group Relative Policy Optimization
(GRPO) by prioritizing high-value samples throughout the optimization process.
Notably, we observe that excessive reinforcement signals can induce visual
hallucinations--a phenomenon we systematically monitor and mitigate through
calibrated reward thresholds throughout the training process. Empirical results
affirm the exceptional capability of R1V2, with benchmark-leading performances
such as 62.6 on OlympiadBench, 79.0 on AIME2024, 63.6 on LiveCodeBench, and
74.0 on MMMU. These results underscore R1V2's superiority over existing
open-source models and demonstrate significant progress in closing the
performance gap with premier proprietary systems, including Gemini 2.5 and
OpenAI o4-mini. The Skywork R1V2 model weights have been publicly released to
promote openness and reproducibility
https://huggingface.co/Skywork/Skywork-R1V2-38B.
Skywork R1V2 enhances multimodal reasoning through a hybrid reinforcement learning approach that balances reward-model guidance and rule-based strategies, improving training efficiency with the Selective Sample Buffer mechanism and mitigating visual hallucinations.
hybrid reinforcement learning reward-model guidance rule-based strategies Selective Sample Buffer (SSB) Vanishing Advantages Group Relative Policy Optimization (GRPO) visual hallucinations OlympiadBench AIME2024 LiveCodeBench MMMU