Uni-MMMU: A Massive Multi-discipline Multimodal Unified Benchmark
Unified multimodal models aim to jointly enable visual understanding and
generation, yet current benchmarks rarely examine their true integration.
Existing evaluations either treat the two abilities in isolation or overlook
tasks that inherently couple them. To address this gap, we present Uni-MMMU, a
comprehensive and discipline-aware benchmark that systematically unfolds the
bidirectional synergy between generation and understanding across eight
reasoning-centric domains, including science, coding, mathematics, and puzzles.
Each task is bidirectionally coupled, demanding models to (i) leverage
conceptual understanding to guide precise visual synthesis, or (ii) utilize
generation as a cognitive scaffold for analytical reasoning. Uni-MMMU
incorporates verifiable intermediate reasoning steps, unique ground truths, and
a reproducible scoring protocol for both textual and visual outputs. Through
extensive evaluation of state-of-the-art unified, generation-only, and
understanding-only models, we reveal substantial performance disparities and
cross-modal dependencies, offering new insights into when and how these
abilities reinforce one another, and establishing a reliable foundation for
advancing unified models.
68f05f6336f8b025381e19c3
Kai Zou
claimed_verified
2025-10-16T06:35:13.604Z
647993d9f966f086918da59e
https://cdn-avatars.huggingface.co/v1/production/uploads/647993d9f966f086918da59e/NDxz3PEpo3srZQNhwT7Qf.jpeg
kzou
user
jackyhate
68f05f6336f8b025381e19c4
Ziqi Huang
68f05f6336f8b025381e19c5
Yuhao Dong
68f05f6336f8b025381e19c6
Shulin Tian
68f05f6336f8b025381e19c7
Dian Zheng
68f05f6336f8b025381e19c8
Hongbo Liu
68f05f6336f8b025381e19c9
Jingwen He
68f05f6336f8b025381e19ca
Bin Liu
68f05f6336f8b025381e19cb
Yu Qiao
68f05f6336f8b025381e19cc
Ziwei Liu
Uni-MMMU: A Massive Multi-discipline Multimodal Unified Benchmark
Unified multimodal models aim to jointly enable visual understanding and
generation, yet current benchmarks rarely examine their true integration.
Existing evaluations either treat the two abilities in isolation or overlook
tasks that inherently couple them. To address this gap, we present Uni-MMMU, a
comprehensive and discipline-aware benchmark that systematically unfolds the
bidirectional synergy between generation and understanding across eight
reasoning-centric domains, including science, coding, mathematics, and puzzles.
Each task is bidirectionally coupled, demanding models to (i) leverage
conceptual understanding to guide precise visual synthesis, or (ii) utilize
generation as a cognitive scaffold for analytical reasoning. Uni-MMMU
incorporates verifiable intermediate reasoning steps, unique ground truths, and
a reproducible scoring protocol for both textual and visual outputs. Through
extensive evaluation of state-of-the-art unified, generation-only, and
understanding-only models, we reveal substantial performance disparities and
cross-modal dependencies, offering new insights into when and how these
abilities reinforce one another, and establishing a reliable foundation for
advancing unified models.
Uni-MMMU is a benchmark that evaluates the bidirectional synergy between visual understanding and generation across multiple domains, providing insights into their integration and performance.
unified multimodal models visual understanding visual generation bidirectional synergy reasoning-centric domains conceptual understanding precise visual synthesis cognitive scaffold analytical reasoning verifiable intermediate reasoning steps unique ground truths reproducible scoring protocol unified models generation-only models understanding-only models cross-modal dependencies