RealUnify: Do Unified Models Truly Benefit from Unification? A
  Comprehensive Benchmark
The integration of visual understanding and generation into unified
multimodal models represents a significant stride toward general-purpose AI.
However, a fundamental question remains unanswered by existing benchmarks: does
this architectural unification actually enable synergetic interaction between
the constituent capabilities? Existing evaluation paradigms, which primarily
assess understanding and generation in isolation, are insufficient for
determining whether a unified model can leverage its understanding to enhance
its generation, or use generative simulation to facilitate deeper
comprehension. To address this critical gap, we introduce RealUnify, a
benchmark specifically designed to evaluate bidirectional capability synergy.
RealUnify comprises 1,000 meticulously human-annotated instances spanning 10
categories and 32 subtasks. It is structured around two core axes: 1)
Understanding Enhances Generation, which requires reasoning (e.g., commonsense,
logic) to guide image generation, and 2) Generation Enhances Understanding,
which necessitates mental simulation or reconstruction (e.g., of transformed or
disordered visual inputs) to solve reasoning tasks. A key contribution is our
dual-evaluation protocol, which combines direct end-to-end assessment with a
diagnostic stepwise evaluation that decomposes tasks into distinct
understanding and generation phases. This protocol allows us to precisely
discern whether performance bottlenecks stem from deficiencies in core
abilities or from a failure to integrate them. Through large-scale evaluations
of 12 leading unified models and 6 specialized baselines, we find that current
unified models still struggle to achieve effective synergy, indicating that
architectural unification alone is insufficient. These results highlight the
need for new training strategies and inductive biases to fully unlock the
potential of unified modeling.
68db4c16d2bf1f4b15ec744e
Yang Shi
claimed_verified
2025-09-30T06:29:34.746Z
673c7319d11b1c2e246ead9c
https://cdn-avatars.huggingface.co/v1/production/uploads/673c7319d11b1c2e246ead9c/IjFIO--N7Hm_BOEafhEQv.jpeg
Yang Shi
user
DogNeverSleep
68db4c16d2bf1f4b15ec744f
Yuhao Dong
claimed_verified
2025-09-30T10:18:06.242Z
652965773a416e1f2173443b
https://cdn-avatars.huggingface.co/v1/production/uploads/652965773a416e1f2173443b/y9MB8YgHzbwCXAc4EI9T3.jpeg
Yuhao Dong
user
THUdyh
68db4c16d2bf1f4b15ec7450
Yue Ding
68db4c16d2bf1f4b15ec7451
Yuran Wang
68db4c16d2bf1f4b15ec7452
Xuanyu Zhu
68db4c16d2bf1f4b15ec7453
Sheng Zhou
68db4c16d2bf1f4b15ec7454
Wenting Liu
68db4c16d2bf1f4b15ec7455
Haochen Tian
68db4c16d2bf1f4b15ec7456
Rundong Wang
68db4c16d2bf1f4b15ec7457
Huanqian Wang
68db4c16d2bf1f4b15ec7458
Zuyan Liu
68db4c16d2bf1f4b15ec7459
Bohan Zeng
68db4c16d2bf1f4b15ec745a
Ruizhe Chen
68db4c16d2bf1f4b15ec745b
Qixun Wang
68db4c16d2bf1f4b15ec745c
Zhuoran Zhang
68db4c16d2bf1f4b15ec745d
Xinlong Chen
claimed_verified
2025-10-14T07:32:56.965Z
66650d38b52f0890724f3b07
/avatars/c25a365bff4985ebb71c96dd097b804f.svg
Xinlong Chen
user
XinlongChen
68db4c16d2bf1f4b15ec745e
Chengzhuo Tong
68db4c16d2bf1f4b15ec745f
Bozhou Li
claimed_verified
2025-09-30T07:12:22.595Z
661e62c6bac5d981f886f77b
/avatars/f1eb51ed4499ca434c8939573dfbd5e2.svg
Bozhou Li
user
zooblastlbz
68db4c16d2bf1f4b15ec7460
Chaoyou Fu
68db4c16d2bf1f4b15ec7461
Qiang Liu
68db4c16d2bf1f4b15ec7462
Haotian Wang
68db4c16d2bf1f4b15ec7463
Wenjing Yang
68db4c16d2bf1f4b15ec7464
Yuanxing Zhang
68db4c16d2bf1f4b15ec7465
Pengfei Wan
68db4c16d2bf1f4b15ec7466
Yi-Fan Zhang
68db4c16d2bf1f4b15ec7467
Ziwei Liu
RealUnify: Do Unified Models Truly Benefit from Unification? A
  Comprehensive Benchmark
The integration of visual understanding and generation into unified
multimodal models represents a significant stride toward general-purpose AI.
However, a fundamental question remains unanswered by existing benchmarks: does
this architectural unification actually enable synergetic interaction between
the constituent capabilities? Existing evaluation paradigms, which primarily
assess understanding and generation in isolation, are insufficient for
determining whether a unified model can leverage its understanding to enhance
its generation, or use generative simulation to facilitate deeper
comprehension. To address this critical gap, we introduce RealUnify, a
benchmark specifically designed to evaluate bidirectional capability synergy.
RealUnify comprises 1,000 meticulously human-annotated instances spanning 10
categories and 32 subtasks. It is structured around two core axes: 1)
Understanding Enhances Generation, which requires reasoning (e.g., commonsense,
logic) to guide image generation, and 2) Generation Enhances Understanding,
which necessitates mental simulation or reconstruction (e.g., of transformed or
disordered visual inputs) to solve reasoning tasks. A key contribution is our
dual-evaluation protocol, which combines direct end-to-end assessment with a
diagnostic stepwise evaluation that decomposes tasks into distinct
understanding and generation phases. This protocol allows us to precisely
discern whether performance bottlenecks stem from deficiencies in core
abilities or from a failure to integrate them. Through large-scale evaluations
of 12 leading unified models and 6 specialized baselines, we find that current
unified models still struggle to achieve effective synergy, indicating that
architectural unification alone is insufficient. These results highlight the
need for new training strategies and inductive biases to fully unlock the
potential of unified modeling.
RealUnify evaluates the bidirectional synergy between understanding and generation in unified multimodal models, revealing that current models lack effective integration despite architectural unification.
multimodal models RealUnify bidirectional capability synergy understanding enhances generation generation enhances understanding dual-evaluation protocol unified models specialized baselines