Skywork-VL Reward: An Effective Reward Model for Multimodal
  Understanding and Reasoning
We propose Skywork-VL Reward, a multimodal reward model that provides reward
signals for both multimodal understanding and reasoning tasks. Our technical
approach comprises two key components: First, we construct a large-scale
multimodal preference dataset that covers a wide range of tasks and scenarios,
with responses collected from both standard vision-language models (VLMs) and
advanced VLM reasoners. Second, we design a reward model architecture based on
Qwen2.5-VL-7B-Instruct, integrating a reward head and applying multi-stage
fine-tuning using pairwise ranking loss on pairwise preference data.
Experimental evaluations show that Skywork-VL Reward achieves state-of-the-art
results on multimodal VL-RewardBench and exhibits competitive performance on
the text-only RewardBench benchmark. Furthermore, preference data constructed
based on our Skywork-VL Reward proves highly effective for training Mixed
Preference Optimization (MPO), leading to significant improvements in
multimodal reasoning capabilities. Our results underscore Skywork-VL Reward as
a significant advancement toward general-purpose, reliable reward models for
multimodal alignment. Our model has been publicly released to promote
transparency and reproducibility.
6822cf4c3b7392637edd756a
Xiaokun Wang
claimed_verified
2025-05-13T10:27:21.212Z
62be9b5aae56e75e4d689e7c
/avatars/6772bc09d6eeb4e86b1210481be91720.svg
wangxiaokun
user
shawn0wang
6822cf4c3b7392637edd756b
Chris
claimed_verified
2025-05-13T10:27:19.259Z
620f5a1c3f76c50e6458a9b6
https://cdn-avatars.huggingface.co/v1/production/uploads/620f5a1c3f76c50e6458a9b6/pXh_f5F0UvufxuUa-eS-v.jpeg
Peiyu Wang
user
OrlandoHugBot
6822cf4c3b7392637edd756c
Jiangbo Pei
6822cf4c3b7392637edd756d
Wei Shen
6822cf4c3b7392637edd756e
Yi Peng
6822cf4c3b7392637edd756f
Yunzhuo Hao
6822cf4c3b7392637edd7570
Weijie Qiu
claimed_verified
2025-06-04T09:56:35.421Z
660aab2c878289c5b34f9e97
https://cdn-avatars.huggingface.co/v1/production/uploads/660aab2c878289c5b34f9e97/yxx1-lR8x5o6KaEpZDXQq.jpeg
weijie qiu
user
qiuwj
6822cf4c3b7392637edd7571
Ai Jian
6822cf4c3b7392637edd7572
Tianyidan Xie
6822cf4c3b7392637edd7573
Xuchen Song
6822cf4c3b7392637edd7574
Yang Liu
6822cf4c3b7392637edd7575
Yahui Zhou
Skywork-VL Reward: An Effective Reward Model for Multimodal
  Understanding and Reasoning
We propose Skywork-VL Reward, a multimodal reward model that provides reward
signals for both multimodal understanding and reasoning tasks. Our technical
approach comprises two key components: First, we construct a large-scale
multimodal preference dataset that covers a wide range of tasks and scenarios,
with responses collected from both standard vision-language models (VLMs) and
advanced VLM reasoners. Second, we design a reward model architecture based on
Qwen2.5-VL-7B-Instruct, integrating a reward head and applying multi-stage
fine-tuning using pairwise ranking loss on pairwise preference data.
Experimental evaluations show that Skywork-VL Reward achieves state-of-the-art
results on multimodal VL-RewardBench and exhibits competitive performance on
the text-only RewardBench benchmark. Furthermore, preference data constructed
based on our Skywork-VL Reward proves highly effective for training Mixed
Preference Optimization (MPO), leading to significant improvements in
multimodal reasoning capabilities. Our results underscore Skywork-VL Reward as
a significant advancement toward general-purpose, reliable reward models for
multimodal alignment. Our model has been publicly released to promote
transparency and reproducibility.
Skywork-VL Reward is a multimodal reward model that uses a large-scale preference dataset and Qwen2.5-VL-7B-Instruct architecture to achieve state-of-the-art performance in multimodal reasoning.
multimodal reward model multimodal understanding Qwen2.5-VL-7B-Instruct reward head multi-stage fine-tuning pairwise ranking loss multimodal VL-RewardBench text-only RewardBench Mixed Preference Optimization multimodal reasoning capabilities general-purpose reward models multimodal alignment