OneIG-Bench: Omni-dimensional Nuanced Evaluation for Image Generation
Text-to-image (T2I) models have garnered significant attention for generating
high-quality images aligned with text prompts. However, rapid T2I model
advancements reveal limitations in early benchmarks, lacking comprehensive
evaluations, for example, the evaluation on reasoning, text rendering and
style. Notably, recent state-of-the-art models, with their rich knowledge
modeling capabilities, show promising results on the image generation problems
requiring strong reasoning ability, yet existing evaluation systems have not
adequately addressed this frontier. To systematically address these gaps, we
introduce OneIG-Bench, a meticulously designed comprehensive benchmark
framework for fine-grained evaluation of T2I models across multiple dimensions,
including prompt-image alignment, text rendering precision, reasoning-generated
content, stylization, and diversity. By structuring the evaluation, this
benchmark enables in-depth analysis of model performance, helping researchers
and practitioners pinpoint strengths and bottlenecks in the full pipeline of
image generation. Specifically, OneIG-Bench enables flexible evaluation by
allowing users to focus on a particular evaluation subset. Instead of
generating images for the entire set of prompts, users can generate images only
for the prompts associated with the selected dimension and complete the
corresponding evaluation accordingly. Our codebase and dataset are now publicly
available to facilitate reproducible evaluation studies and cross-model
comparisons within the T2I research community.
684792f03ec10bdd8ab4de06
Jingjing Chang
684792f03ec10bdd8ab4de07
Yixiao Fang
claimed_verified
2025-06-10T08:44:54.679Z
647469b9a51711a3b58bda2b
https://cdn-avatars.huggingface.co/v1/production/uploads/647469b9a51711a3b58bda2b/yeDf8Sa8IDEQyney1dGC9.jpeg
Yixiao Fang
user
fangyixiao
684792f03ec10bdd8ab4de08
Peng Xing
684792f03ec10bdd8ab4de09
Shuhan Wu
684792f03ec10bdd8ab4de0a
Wei Cheng
claimed_verified
2025-06-10T08:45:23.322Z
64b914c8ace99c0723ad83a9
https://cdn-avatars.huggingface.co/v1/production/uploads/64b914c8ace99c0723ad83a9/B4gxNByeVY_xaOcjwiN1j.jpeg
Wei Cheng
user
wchengad
684792f03ec10bdd8ab4de0b
Rui Wang
684792f03ec10bdd8ab4de0c
Xianfang Zeng
684792f03ec10bdd8ab4de0d
Gang Yu
claimed_verified
2025-06-11T08:36:09.701Z
63417332c5565a4b8d43a0d8
https://cdn-avatars.huggingface.co/v1/production/uploads/63417332c5565a4b8d43a0d8/MmnYG7Wu2Z_lqCBvTfJmy.png
Gang Yu
user
skicy
684792f03ec10bdd8ab4de0e
Hai-Bao Chen
OneIG-Bench: Omni-dimensional Nuanced Evaluation for Image Generation
Text-to-image (T2I) models have garnered significant attention for generating
high-quality images aligned with text prompts. However, rapid T2I model
advancements reveal limitations in early benchmarks, lacking comprehensive
evaluations, for example, the evaluation on reasoning, text rendering and
style. Notably, recent state-of-the-art models, with their rich knowledge
modeling capabilities, show promising results on the image generation problems
requiring strong reasoning ability, yet existing evaluation systems have not
adequately addressed this frontier. To systematically address these gaps, we
introduce OneIG-Bench, a meticulously designed comprehensive benchmark
framework for fine-grained evaluation of T2I models across multiple dimensions,
including prompt-image alignment, text rendering precision, reasoning-generated
content, stylization, and diversity. By structuring the evaluation, this
benchmark enables in-depth analysis of model performance, helping researchers
and practitioners pinpoint strengths and bottlenecks in the full pipeline of
image generation. Specifically, OneIG-Bench enables flexible evaluation by
allowing users to focus on a particular evaluation subset. Instead of
generating images for the entire set of prompts, users can generate images only
for the prompts associated with the selected dimension and complete the
corresponding evaluation accordingly. Our codebase and dataset are now publicly
available to facilitate reproducible evaluation studies and cross-model
comparisons within the T2I research community.
OneIG-Bench is a comprehensive benchmark framework for evaluating text-to-image models across multiple dimensions including reasoning, text rendering, and diversity.
text-to-image (T2I) models prompt-image alignment text rendering precision reasoning-generated content stylization diversity