Envisioning Beyond the Pixels: Benchmarking Reasoning-Informed Visual Editing
Xiangyu Zhao, Peiyuan Zhang, Kexian Tang, Xiaorong Zhu, Hao Li, Wenhao Chai, Zicheng Zhang, Renqiu Xia, Guangtao Zhai, Junchi Yan, Hua Yang, Xue Yang, Haodong Duan
Large Multi-modality Models (LMMs) have made significant progress in visual understanding and generation, but they still face challenges in General Visual Editing, particularly in following complex instructions, preserving appearance consistency, and supporting flexible input formats. To study this gap, we introduce \textbf{RISEBench}, the first benchmark for evaluating **R**easoning-**I**nformed vi**S**ual **E**diting (**RISE**). RISEBench focuses on four key reasoning categories: *Temporal*, *Causal*, *Spatial*, and *Logical Reasoning*. We curate high-quality test cases for each category and propose an robust evaluation framework that assesses *Instruction Reasoning*, *Appearance Consistency*, and *Visual Plausibility* with both human judges and the LMM-as-a-judge approach. We conducted experiments evaluating eight prominent visual editing models, comprising both open-source and proprietary models. The evaluation results demonstrate that current models face significant challenges in reasoning-based editing tasks. Even the most powerful model evaluated, GPT-4o-Image, achieves an accuracy of merely 28.8\%. RISEBench effectively highlights the limitations of contemporary editing models, provides valuable insights, and indicates potential future directions for the field of reasoning-aware visual editing.