Easier Painting Than Thinking: Can Text-to-Image Models Set the Stage,
  but Not Direct the Play?
Text-to-image (T2I) generation aims to synthesize images from textual
prompts, which jointly specify what must be shown and imply what can be
inferred, thereby corresponding to two core capabilities: composition and
reasoning. However, with the emerging advances of T2I models in reasoning
beyond composition, existing benchmarks reveal clear limitations in providing
comprehensive evaluations across and within these capabilities. Meanwhile,
these advances also enable models to handle more complex prompts, whereas
current benchmarks remain limited to low scene density and simplified
one-to-one reasoning. To address these limitations, we propose T2I-CoReBench, a
comprehensive and complex benchmark that evaluates both composition and
reasoning capabilities of T2I models. To ensure comprehensiveness, we structure
composition around scene graph elements (instance, attribute, and relation) and
reasoning around the philosophical framework of inference (deductive,
inductive, and abductive), formulating a 12-dimensional evaluation taxonomy. To
increase complexity, driven by the inherent complexities of real-world
scenarios, we curate each prompt with high compositional density for
composition and multi-step inference for reasoning. We also pair each prompt
with a checklist that specifies individual yes/no questions to assess each
intended element independently to facilitate fine-grained and reliable
evaluation. In statistics, our benchmark comprises 1,080 challenging prompts
and around 13,500 checklist questions. Experiments across 27 current T2I models
reveal that their composition capability still remains limited in complex
high-density scenarios, while the reasoning capability lags even further behind
as a critical bottleneck, with all models struggling to infer implicit elements
from prompts. Our project page: https://t2i-corebench.github.io/.
68b93427d43cadaf7a688bd1
Ouxiang Li
claimed_verified
2025-09-04T08:42:54.336Z
666c06ca9ed9e91df03e7e27
/avatars/3101ad4f1a3a243b75080d06c5e59e9c.svg
Ouxiang Li
user
lioooox
68b93427d43cadaf7a688bd2
Yuan Wang
68b93427d43cadaf7a688bd3
Xinting Hu
68b93427d43cadaf7a688bd4
Huijuan Huang
68b93427d43cadaf7a688bd5
Rui Chen
68b93427d43cadaf7a688bd6
Jiarong Ou
68b93427d43cadaf7a688bd7
Xin Tao
68b93427d43cadaf7a688bd8
Pengfei Wan
68b93427d43cadaf7a688bd9
Fuli Feng
Easier Painting Than Thinking: Can Text-to-Image Models Set the Stage,
  but Not Direct the Play?
Text-to-image (T2I) generation aims to synthesize images from textual
prompts, which jointly specify what must be shown and imply what can be
inferred, thereby corresponding to two core capabilities: composition and
reasoning. However, with the emerging advances of T2I models in reasoning
beyond composition, existing benchmarks reveal clear limitations in providing
comprehensive evaluations across and within these capabilities. Meanwhile,
these advances also enable models to handle more complex prompts, whereas
current benchmarks remain limited to low scene density and simplified
one-to-one reasoning. To address these limitations, we propose T2I-CoReBench, a
comprehensive and complex benchmark that evaluates both composition and
reasoning capabilities of T2I models. To ensure comprehensiveness, we structure
composition around scene graph elements (instance, attribute, and relation) and
reasoning around the philosophical framework of inference (deductive,
inductive, and abductive), formulating a 12-dimensional evaluation taxonomy. To
increase complexity, driven by the inherent complexities of real-world
scenarios, we curate each prompt with high compositional density for
composition and multi-step inference for reasoning. We also pair each prompt
with a checklist that specifies individual yes/no questions to assess each
intended element independently to facilitate fine-grained and reliable
evaluation. In statistics, our benchmark comprises 1,080 challenging prompts
and around 13,500 checklist questions. Experiments across 27 current T2I models
reveal that their composition capability still remains limited in complex
high-density scenarios, while the reasoning capability lags even further behind
as a critical bottleneck, with all models struggling to infer implicit elements
from prompts. Our project page: https://t2i-corebench.github.io/.
T2I-CoReBench is a benchmark that evaluates the composition and reasoning capabilities of text-to-image models using a comprehensive and complex set of prompts and checklist questions.
text-to-image (T2I) generation composition reasoning scene graph elements inference deductive inductive abductive evaluation taxonomy compositional density multi-step inference checklist questions