VisionReward: Fine-Grained Multi-Dimensional Human Preference Learning
  for Image and Video Generation
We present a general strategy to aligning visual generation models -- both
image and video generation -- with human preference. To start with, we build
VisionReward -- a fine-grained and multi-dimensional reward model. We decompose
human preferences in images and videos into multiple dimensions, each
represented by a series of judgment questions, linearly weighted and summed to
an interpretable and accurate score. To address the challenges of video quality
assessment, we systematically analyze various dynamic features of videos, which
helps VisionReward surpass VideoScore by 17.2% and achieve top performance for
video preference prediction. Based on VisionReward, we develop a
multi-objective preference learning algorithm that effectively addresses the
issue of confounding factors within preference data. Our approach significantly
outperforms existing image and video scoring methods on both machine metrics
and human evaluation. All code and datasets are provided at
https://github.com/THUDM/VisionReward.
677a6fb5deb62f25f4209c2a
Jiazheng Xu
claimed_verified
2025-01-06T08:00:26.452Z
62d7b131f6e8ba66107af761
/avatars/f1c5df47aef69c824fd166722df8f670.svg
Jiazheng Xu
user
xujz0703
677a6fb5deb62f25f4209c2b
Yu Huang
677a6fb5deb62f25f4209c2c
Jiale Cheng
admin_assigned
2025-01-06T12:41:34.810Z
627626d42d26ac639e56f565
/avatars/805c5f909f52656345b8bde486c9fa8f.svg
Jiale Cheng
user
CCCCCC
677a6fb5deb62f25f4209c2d
Yuanming Yang
admin_assigned
2025-01-06T12:41:48.700Z
659b6c50b0f43ed69fe09d56
/avatars/8ea56c56263595a9f7555f2c2520641a.svg
杨远明
user
yangyuanming
677a6fb5deb62f25f4209c2e
Jiajun Xu
677a6fb5deb62f25f4209c2f
Yuan Wang
claimed_verified
2025-07-02T08:55:31.558Z
6864ef64d4dde09fbabad95a
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/sbEmJxiJg4-8rrXDqUuFy.jpeg
Yuan Wang
user
traveler2333
677a6fb5deb62f25f4209c30
Wenbo Duan
677a6fb5deb62f25f4209c31
Shen Yang
677a6fb5deb62f25f4209c32
Qunlin Jin
admin_assigned
2025-01-06T12:42:43.541Z
65007e0870b6b05c5ad94a62
/avatars/c1ff713db5748db121738c601f8add85.svg
Jin Qunlin
user
kimtorch
677a6fb5deb62f25f4209c33
Shurun Li
677a6fb5deb62f25f4209c34
Jiayan Teng
admin_assigned
2025-01-06T12:43:16.822Z
65228733377bffdc59a10117
/avatars/6eec07553658ab22f8058caa0bfbed49.svg
tengjiayan
user
tengjiayan
677a6fb5deb62f25f4209c35
Zhuoyi Yang
admin_assigned
2025-01-06T12:43:40.186Z
6466d1640ed2f7a8cba87503
/avatars/652746e63dfeb5154ae7d34039d1a485.svg
Zhuoyi Yang
user
zyyangzy
677a6fb5deb62f25f4209c36
Wendi Zheng
admin_assigned
2025-01-06T12:44:00.832Z
650aaaf1634e02df56dfd231
/avatars/4643ab23721d4ed6aeb1ebbc717adc43.svg
Wendi Zheng
user
zwd125
677a6fb5deb62f25f4209c37
Xiao Liu
677a6fb5deb62f25f4209c38
Ming Ding
677a6fb5deb62f25f4209c39
Xiaohan Zhang
677a6fb5deb62f25f4209c3a
Xiaotao Gu
claimed_verified
2025-07-02T09:25:01.267Z
6438bca4a5e10f6d58694b47
/avatars/3aeb25fbc73c5cab1265e13d11adfb76.svg
Xiaotao Gu
user
xgeric
677a6fb5deb62f25f4209c3b
Shiyu Huang
admin_assigned
2025-01-06T12:39:28.802Z
6406db5cd684369027166986
https://cdn-avatars.huggingface.co/v1/production/uploads/6406db5cd684369027166986/Zl-orrGcbY0RbfjfKszn1.jpeg
Shiyu Huang
user
ShiyuHuang
677a6fb5deb62f25f4209c3c
Minlie Huang
677a6fb5deb62f25f4209c3d
Jie Tang
admin_assigned
2025-01-06T12:38:45.096Z
640dff05474aa6f89556677e
/avatars/1b4591c7322d649c797b3125148f1915.svg
Jie Tang
user
jerytang
677a6fb5deb62f25f4209c3e
Yuxiao Dong
admin_assigned
2025-01-06T12:38:37.430Z
640e73bdfdeaae1390857b62
/avatars/cd6779e30f716002a7838ed93d5c0754.svg
Yuxiao Dong
user
yuxiaod
VisionReward: Fine-Grained Multi-Dimensional Human Preference Learning
  for Image and Video Generation
We present a general strategy to aligning visual generation models -- both
image and video generation -- with human preference. To start with, we build
VisionReward -- a fine-grained and multi-dimensional reward model. We decompose
human preferences in images and videos into multiple dimensions, each
represented by a series of judgment questions, linearly weighted and summed to
an interpretable and accurate score. To address the challenges of video quality
assessment, we systematically analyze various dynamic features of videos, which
helps VisionReward surpass VideoScore by 17.2% and achieve top performance for
video preference prediction. Based on VisionReward, we develop a
multi-objective preference learning algorithm that effectively addresses the
issue of confounding factors within preference data. Our approach significantly
outperforms existing image and video scoring methods on both machine metrics
and human evaluation. All code and datasets are provided at
https://github.com/THUDM/VisionReward.
A reward model with multi-dimensional analysis aligns visual generation models with human preferences, surpassing existing methods in image and video scoring.
VisionReward reward model judgment questions multi-dimensional preference learning confounding factors VideoScore preference prediction