UniGenBench++: A Unified Semantic Evaluation Benchmark for Text-to-Image
  Generation
Recent progress in text-to-image (T2I) generation underscores the importance
of reliable benchmarks in evaluating how accurately generated images reflect
the semantics of their textual prompt. However, (1) existing benchmarks lack
the diversity of prompt scenarios and multilingual support, both essential for
real-world applicability; (2) they offer only coarse evaluations across primary
dimensions, covering a narrow range of sub-dimensions, and fall short in
fine-grained sub-dimension assessment. To address these limitations, we
introduce UniGenBench++, a unified semantic assessment benchmark for T2I
generation. Specifically, it comprises 600 prompts organized hierarchically to
ensure both coverage and efficiency: (1) spans across diverse real-world
scenarios, i.e., 5 main prompt themes and 20 subthemes; (2) comprehensively
probes T2I models' semantic consistency over 10 primary and 27 sub evaluation
criteria, with each prompt assessing multiple testpoints. To rigorously assess
model robustness to variations in language and prompt length, we provide both
English and Chinese versions of each prompt in short and long forms. Leveraging
the general world knowledge and fine-grained image understanding capabilities
of a closed-source Multi-modal Large Language Model (MLLM), i.e.,
Gemini-2.5-Pro, an effective pipeline is developed for reliable benchmark
construction and streamlined model assessment. Moreover, to further facilitate
community use, we train a robust evaluation model that enables offline
assessment of T2I model outputs. Through comprehensive benchmarking of both
open- and closed-sourced T2I models, we systematically reveal their strengths
and weaknesses across various aspects.
68f839917669bcaeecce0c98
Yibin Wang
claimed_verified
2025-10-22T15:44:11.252Z
654c6845bac6e6e49895a5b5
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/KXQaAxulqr8jNBSpEaYM4.png
SII-Yibin Wang
user
CodeGoat24
68f839917669bcaeecce0c99
Zhimin Li
68f839917669bcaeecce0c9a
Yuhang Zang
claimed_verified
2025-10-22T15:44:14.239Z
63859cf3b2906edaf83af9f0
https://cdn-avatars.huggingface.co/v1/production/uploads/63859cf3b2906edaf83af9f0/kajwuVzd4pDucSPlwghxo.png
Yuhang Zang
user
yuhangzang
68f839917669bcaeecce0c9b
Jiazi Bu
68f839917669bcaeecce0c9c
Yujie Zhou
68f839917669bcaeecce0c9d
Yi Xin
68f839917669bcaeecce0c9e
Junjun He
68f839917669bcaeecce0c9f
Chunyu Wang
68f839917669bcaeecce0ca0
Qinglin Lu
68f839917669bcaeecce0ca1
Cheng Jin
68f839917669bcaeecce0ca2
Jiaqi Wang
UniGenBench++: A Unified Semantic Evaluation Benchmark for Text-to-Image
  Generation
Recent progress in text-to-image (T2I) generation underscores the importance
of reliable benchmarks in evaluating how accurately generated images reflect
the semantics of their textual prompt. However, (1) existing benchmarks lack
the diversity of prompt scenarios and multilingual support, both essential for
real-world applicability; (2) they offer only coarse evaluations across primary
dimensions, covering a narrow range of sub-dimensions, and fall short in
fine-grained sub-dimension assessment. To address these limitations, we
introduce UniGenBench++, a unified semantic assessment benchmark for T2I
generation. Specifically, it comprises 600 prompts organized hierarchically to
ensure both coverage and efficiency: (1) spans across diverse real-world
scenarios, i.e., 5 main prompt themes and 20 subthemes; (2) comprehensively
probes T2I models' semantic consistency over 10 primary and 27 sub evaluation
criteria, with each prompt assessing multiple testpoints. To rigorously assess
model robustness to variations in language and prompt length, we provide both
English and Chinese versions of each prompt in short and long forms. Leveraging
the general world knowledge and fine-grained image understanding capabilities
of a closed-source Multi-modal Large Language Model (MLLM), i.e.,
Gemini-2.5-Pro, an effective pipeline is developed for reliable benchmark
construction and streamlined model assessment. Moreover, to further facilitate
community use, we train a robust evaluation model that enables offline
assessment of T2I model outputs. Through comprehensive benchmarking of both
open- and closed-sourced T2I models, we systematically reveal their strengths
and weaknesses across various aspects.
UniGenBench++ is a comprehensive benchmark for text-to-image generation that evaluates semantic consistency across diverse scenarios and languages using a hierarchical prompt structure and a robust evaluation pipeline.
text-to-image generation semantic assessment benchmark prompt scenarios multilingual support semantic consistency evaluation criteria Multi-modal Large Language Model MLLM Gemini-2.5-Pro offline assessment