WISE: A World Knowledge-Informed Semantic Evaluation for Text-to-Image
  Generation
Text-to-Image (T2I) models are capable of generating high-quality artistic
creations and visual content. However, existing research and evaluation
standards predominantly focus on image realism and shallow text-image
alignment, lacking a comprehensive assessment of complex semantic understanding
and world knowledge integration in text to image generation. To address this
challenge, we propose WISE, the first benchmark specifically
designed for World Knowledge-Informed Semantic
Evaluation. WISE moves beyond simple word-pixel mapping by
challenging models with 1000 meticulously crafted prompts across 25 sub-domains
in cultural common sense, spatio-temporal reasoning, and natural science. To
overcome the limitations of traditional CLIP metric, we introduce
WiScore, a novel quantitative metric for assessing knowledge-image
alignment. Through comprehensive testing of 20 models (10 dedicated T2I models
and 10 unified multimodal models) using 1,000 structured prompts spanning 25
subdomains, our findings reveal significant limitations in their ability to
effectively integrate and apply world knowledge during image generation,
highlighting critical pathways for enhancing knowledge incorporation and
application in next-generation T2I models. Code and data are available at
https://github.com/PKU-YuanGroup/WISE.
67d01cf289beb7795a51f111
Yuwei Niu
67d01cf289beb7795a51f112
Munan Ning
admin_assigned
2025-03-11T14:56:00.839Z
65e14c28b1a6de8a71e70172
https://cdn-avatars.huggingface.co/v1/production/uploads/65e14c28b1a6de8a71e70172/D097SILGsqoufpp3sG8tV.jpeg
Munan Ning
user
MunanNing
67d01cf289beb7795a51f113
Mengren Zheng
admin_assigned
2025-03-11T14:56:12.978Z
66c16d5d780d735f17bccfdc
/avatars/6bff393b61a78b425d6f429a6d75ad56.svg
Mengren Zheng
user
zmr66z6xx6
67d01cf289beb7795a51f114
Bin Lin
claimed_verified
2025-06-04T09:05:29.423Z
6367a8175bb06007ea099b8f
https://cdn-avatars.huggingface.co/v1/production/uploads/6367a8175bb06007ea099b8f/IjG7HyWyWRlVt_XwRbxRW.jpeg
linbin
user
LanguageBind
67d01cf289beb7795a51f115
Peng Jin
admin_assigned
2025-03-11T14:56:20.825Z
63ad0b04e3b217fb36d36c13
/avatars/5a3715ba20859052ba04c048db9e03c2.svg
Peng Jin
user
Pengjin
67d01cf289beb7795a51f116
Jiaqi Liao
admin_assigned
2025-03-11T14:56:32.257Z
6630b287ce65a66ed8f04eba
/avatars/18ad0e3b453021ea6006e23333e92c12.svg
liaojiaqi
user
ljq940913
67d01cf289beb7795a51f117
Kunpeng Ning
67d01cf289beb7795a51f118
Bin Zhu
admin_assigned
2025-03-11T14:56:47.289Z
6476e69d99a5ce743ccde294
/avatars/12955bcec780ff34a7e78281b45b016d.svg
Bin Zhu
user
binzhu2023
67d01cf289beb7795a51f119
Li Yuan
admin_assigned
2025-03-11T14:56:54.455Z
614030bd8cbdb613b82f36a8
https://cdn-avatars.huggingface.co/v1/production/uploads/1631596713749-noauth.jpeg
Li Yuan
user
LiYuan
WISE: A World Knowledge-Informed Semantic Evaluation for Text-to-Image
  Generation
Text-to-Image (T2I) models are capable of generating high-quality artistic
creations and visual content. However, existing research and evaluation
standards predominantly focus on image realism and shallow text-image
alignment, lacking a comprehensive assessment of complex semantic understanding
and world knowledge integration in text to image generation. To address this
challenge, we propose WISE, the first benchmark specifically
designed for World Knowledge-Informed Semantic
Evaluation. WISE moves beyond simple word-pixel mapping by
challenging models with 1000 meticulously crafted prompts across 25 sub-domains
in cultural common sense, spatio-temporal reasoning, and natural science. To
overcome the limitations of traditional CLIP metric, we introduce
WiScore, a novel quantitative metric for assessing knowledge-image
alignment. Through comprehensive testing of 20 models (10 dedicated T2I models
and 10 unified multimodal models) using 1,000 structured prompts spanning 25
subdomains, our findings reveal significant limitations in their ability to
effectively integrate and apply world knowledge during image generation,
highlighting critical pathways for enhancing knowledge incorporation and
application in next-generation T2I models. Code and data are available at
https://github.com/PKU-YuanGroup/WISE.
WISE is a benchmark for evaluating text-to-image models' integration of world knowledge and semantic understanding through 1,000 prompts across various domains, challenging traditional metrics with WiScore.
Text-to-Image (T2I) semantic understanding world knowledge CLIP metric WiScore multimodal models image generation cultural common sense spatio-temporal reasoning natural science