LLaVA-Critic: Learning to Evaluate Multimodal Models
We introduce LLaVA-Critic, the first open-source large multimodal model (LMM)
designed as a generalist evaluator to assess performance across a wide range of
multimodal tasks. LLaVA-Critic is trained using a high-quality critic
instruction-following dataset that incorporates diverse evaluation criteria and
scenarios. Our experiments demonstrate the model's effectiveness in two key
areas: (1) LMM-as-a-Judge, where LLaVA-Critic provides reliable evaluation
scores, performing on par with or surpassing GPT models on multiple evaluation
benchmarks; and (2) Preference Learning, where it generates reward signals for
preference learning, enhancing model alignment capabilities. This work
underscores the potential of open-source LMMs in self-critique and evaluation,
setting the stage for future research into scalable, superhuman alignment
feedback mechanisms for LMMs.
66ff51ed9e1143bff207d587
Tianyi Xiong
admin_assigned
2024-10-04T08:41:36.661Z
6570977f87a92b76922c9950
https://cdn-avatars.huggingface.co/v1/production/uploads/6570977f87a92b76922c9950/AQGto1w6ugBvH2yCV46YU.jpeg
Tianyi Xiong
user
txiong23
66ff51ed9e1143bff207d588
Xiyao Wang
admin_assigned
2024-10-04T08:44:04.812Z
655fed9fdef5905d38b84af3
/avatars/2cda4182dfd11a1e94743639e62328ea.svg
Xiyao Wang
user
russwang
66ff51ed9e1143bff207d589
Dong Guo
66ff51ed9e1143bff207d58a
Qinghao Ye
claimed_verified
2025-05-22T07:19:34.463Z
64530fc01a57e1179c1fe4c0
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/0lncTpIHXn6suB0p-oSma.jpeg
QinghaoYe
user
MAGAer13
66ff51ed9e1143bff207d58b
Haoqi Fan
66ff51ed9e1143bff207d58c
Quanquan Gu
admin_assigned
2024-10-04T08:41:11.820Z
64c039128e2612254356bba5
/avatars/06cc76feebba0cc80ebb8f4ff86f6d9b.svg
Quanquan Gu
user
thughost
66ff51ed9e1143bff207d58d
Heng Huang
admin_assigned
2024-10-04T08:43:29.377Z
6527bc6b34bf5ece73da426d
/avatars/120739a9ac84e7319d9ea157a63dc547.svg
henghuang
user
henghuang
66ff51ed9e1143bff207d58e
Chunyuan Li
admin_assigned
2024-10-04T08:43:01.990Z
62aba526cae4462c0c6caa0f
/avatars/430560ec2c2547f819225769ab432f30.svg
Chunyuan Li
user
Chunyuan24
LLaVA-Critic: Learning to Evaluate Multimodal Models
We introduce LLaVA-Critic, the first open-source large multimodal model (LMM)
designed as a generalist evaluator to assess performance across a wide range of
multimodal tasks. LLaVA-Critic is trained using a high-quality critic
instruction-following dataset that incorporates diverse evaluation criteria and
scenarios. Our experiments demonstrate the model's effectiveness in two key
areas: (1) LMM-as-a-Judge, where LLaVA-Critic provides reliable evaluation
scores, performing on par with or surpassing GPT models on multiple evaluation
benchmarks; and (2) Preference Learning, where it generates reward signals for
preference learning, enhancing model alignment capabilities. This work
underscores the potential of open-source LMMs in self-critique and evaluation,
setting the stage for future research into scalable, superhuman alignment
feedback mechanisms for LMMs.
LLaVA-Critic, an open-source large multimodal model, effectively evaluates multimodal tasks and provides reliable scores, surpassing GPT models, and enhances preference learning for model alignment.
large multimodal model LMM instruction-following dataset evaluation criteria LMM-as-a-Judge GPT models evaluation benchmarks Preference Learning reward signals model alignment superhuman alignment feedback mechanisms