Skywork-VL Reward: An Effective Reward Model for Multimodal
  Understanding and Reasoning
We propose Skywork-VL Reward, a multimodal reward model that provides reward
signals for both multimodal understanding and reasoning tasks. Our technical
approach comprises two key components: First, we construct a large-scale
multimodal preference dataset that covers a wide range of tasks and scenarios,
with responses collected from both standard vision-language models (VLMs) and
advanced VLM reasoners. Second, we design a reward model architecture based on
Qwen2.5-VL-7B-Instruct, integrating a reward head and applying multi-stage
fine-tuning using pairwise ranking loss on pairwise preference data.
Experimental evaluations show that Skywork-VL Reward achieves state-of-the-art
results on multimodal VL-RewardBench and exhibits competitive performance on
the text-only RewardBench benchmark. Furthermore, preference data constructed
based on our Skywork-VL Reward proves highly effective for training Mixed
Preference Optimization (MPO), leading to significant improvements in
multimodal reasoning capabilities. Our results underscore Skywork-VL Reward as
a significant advancement toward general-purpose, reliable reward models for
multimodal alignment. Our model has been publicly released to promote
transparency and reproducibility.
6822cf4c3b7392637edd756a
Xiaokun Wang
claimed_verified
2025-05-13T10:27:21.212Z
62be9b5aae56e75e4d689e7c
/avatars/6772bc09d6eeb4e86b1210481be91720.svg
wangxiaokun
user
shawn0wang
6822cf4c3b7392637edd756b
Chris
claimed_verified
2025-05-13T10:27:19.259Z
620f5a1c3f76c50e6458a9b6
https://cdn-avatars.huggingface.co/v1/production/uploads/620f5a1c3f76c50e6458a9b6/pXh_f5F0UvufxuUa-eS-v.jpeg
chris
user
OrlandoHugBot
6822cf4c3b7392637edd756c
Jiangbo Pei
6822cf4c3b7392637edd756d
Wei Shen
6822cf4c3b7392637edd756e
Yi Peng
6822cf4c3b7392637edd756f
Yunzhuo Hao
6822cf4c3b7392637edd7570
Weijie Qiu
6822cf4c3b7392637edd7571
Ai Jian
6822cf4c3b7392637edd7572
Tianyidan Xie
6822cf4c3b7392637edd7573
Xuchen Song
6822cf4c3b7392637edd7574
Yang Liu
6822cf4c3b7392637edd7575
Yahui Zhou
Skywork-VL Reward: An Effective Reward Model for Multimodal
  Understanding and Reasoning
We propose Skywork-VL Reward, a multimodal reward model that provides reward
signals for both multimodal understanding and reasoning tasks. Our technical
approach comprises two key components: First, we construct a large-scale
multimodal preference dataset that covers a wide range of tasks and scenarios,
with responses collected from both standard vision-language models (VLMs) and
advanced VLM reasoners. Second, we design a reward model architecture based on
Qwen2.5-VL-7B-Instruct, integrating a reward head and applying multi-stage
fine-tuning using pairwise ranking loss on pairwise preference data.
Experimental evaluations show that Skywork-VL Reward achieves state-of-the-art
results on multimodal VL-RewardBench and exhibits competitive performance on
the text-only RewardBench benchmark. Furthermore, preference data constructed
based on our Skywork-VL Reward proves highly effective for training Mixed
Preference Optimization (MPO), leading to significant improvements in
multimodal reasoning capabilities. Our results underscore Skywork-VL Reward as
a significant advancement toward general-purpose, reliable reward models for
multimodal alignment. Our model has been publicly released to promote
transparency and reproducibility.
Skywork-VL Reward is a multimodal reward model that uses a large-scale preference dataset and Qwen2.5-VL-7B-Instruct architecture to achieve state-of-the-art performance in multimodal reasoning.
multimodal reward model multimodal understanding Qwen2.5-VL-7B-Instruct reward head multi-stage fine-tuning pairwise ranking loss multimodal VL-RewardBench text-only RewardBench Mixed Preference Optimization multimodal reasoning capabilities general-purpose reward models multimodal alignment