GenExam: A Multidisciplinary Text-to-Image Exam
Exams are a fundamental test of expert-level intelligence and require
integrated understanding, reasoning, and generation. Existing exam-style
benchmarks mainly focus on understanding and reasoning tasks, and current
generation benchmarks emphasize the illustration of world knowledge and visual
concepts, neglecting the evaluation of rigorous drawing exams. We introduce
GenExam, the first benchmark for multidisciplinary text-to-image exams,
featuring 1,000 samples across 10 subjects with exam-style prompts organized
under a four-level taxonomy. Each problem is equipped with ground-truth images
and fine-grained scoring points to enable a precise evaluation of semantic
correctness and visual plausibility. Experiments show that even
state-of-the-art models such as GPT-Image-1 and Gemini-2.5-Flash-Image achieve
less than 15% strict scores, and most models yield almost 0%, suggesting the
great challenge of our benchmark. By framing image generation as an exam,
GenExam offers a rigorous assessment of models' ability to integrate knowledge,
reasoning, and generation, providing insights on the path to general AGI.
68cb67a45a7803ff3be42d1e
Zhaokai Wang
claimed_verified
2025-09-18T13:27:10.361Z
665d4b515fdfe8f923e347a7
/avatars/d114b24c02dadfca0a8aee104755a8ec.svg
Zhaokai Wang
user
wzk1015
68cb67a45a7803ff3be42d1f
Penghao Yin
claimed_verified
2025-09-19T06:50:23.920Z
674919baf73677ad66df861e
/avatars/f50544f6196977e37e830d93c378efb5.svg
Penghao
user
pH202411
68cb67a45a7803ff3be42d20
Xiangyu Zhao
68cb67a45a7803ff3be42d21
Changyao Tian
68cb67a45a7803ff3be42d22
Yu Qiao
68cb67a45a7803ff3be42d23
Wenhai Wang
68cb67a45a7803ff3be42d24
Jifeng Dai
68cb67a45a7803ff3be42d25
Gen Luo
GenExam: A Multidisciplinary Text-to-Image Exam
Exams are a fundamental test of expert-level intelligence and require
integrated understanding, reasoning, and generation. Existing exam-style
benchmarks mainly focus on understanding and reasoning tasks, and current
generation benchmarks emphasize the illustration of world knowledge and visual
concepts, neglecting the evaluation of rigorous drawing exams. We introduce
GenExam, the first benchmark for multidisciplinary text-to-image exams,
featuring 1,000 samples across 10 subjects with exam-style prompts organized
under a four-level taxonomy. Each problem is equipped with ground-truth images
and fine-grained scoring points to enable a precise evaluation of semantic
correctness and visual plausibility. Experiments show that even
state-of-the-art models such as GPT-Image-1 and Gemini-2.5-Flash-Image achieve
less than 15% strict scores, and most models yield almost 0%, suggesting the
great challenge of our benchmark. By framing image generation as an exam,
GenExam offers a rigorous assessment of models' ability to integrate knowledge,
reasoning, and generation, providing insights on the path to general AGI.
GenExam is a benchmark for evaluating text-to-image generation in exam-style settings across multiple disciplines, highlighting the challenges in integrating knowledge, reasoning, and generation.
text-to-image GenExam multidisciplinary exam-style prompts ground-truth images fine-grained scoring semantic correctness visual plausibility GPT-Image-1 Gemini-2.5-Flash-Image general AGI