Envisioning Beyond the Pixels: Benchmarking Reasoning-Informed Visual
  Editing
Large Multi-modality Models (LMMs) have made significant progress in visual
understanding and generation, but they still face challenges in General Visual
Editing, particularly in following complex instructions, preserving appearance
consistency, and supporting flexible input formats. To address this gap, we
introduce RISEBench, the first benchmark for evaluating Reasoning-Informed
viSual Editing (RISE). RISEBench focuses on four key reasoning types: Temporal,
Causal, Spatial, and Logical Reasoning. We curate high-quality test cases for
each category and propose an evaluation framework that assesses Instruction
Reasoning, Appearance Consistency, and Visual Plausibility with both human
judges and an LMM-as-a-judge approach. Our experiments reveal that while
GPT-4o-Native significantly outperforms other open-source and proprietary
models, even this state-of-the-art system struggles with logical reasoning
tasks, highlighting an area that remains underexplored. As an initial effort,
RISEBench aims to provide foundational insights into reasoning-aware visual
editing and to catalyze future research. Though still in its early stages, we
are committed to continuously expanding and refining the benchmark to support
more comprehensive, reliable, and scalable evaluations of next-generation
multimodal systems. Our code and data will be released at
https://github.com/PhoenixZ810/RISEBench.
67ef4be0985aa66b67021ddc
Xiangyu Zhao
claimed_verified
2025-04-04T07:09:07.389Z
6530e62f536dbca918e71c3e
/avatars/efc93bc767e561c6c6d429f65c23382d.svg
Xiangyu Z
user
PhoenixZ
67ef4be0985aa66b67021ddd
Peiyuan Zhang
claimed_verified
2025-04-04T07:09:11.816Z
6710be3e6d1b33cf24417e38
/avatars/f60bc9a67bb58f5997cbcc28cb93c079.svg
zpy
user
zpy777
67ef4be0985aa66b67021dde
Kexian Tang
admin_assigned
2025-04-04T07:21:18.584Z
662516d72419feed62fb3a0a
/avatars/24c4157829e70a4e346aa984885aa5ad.svg
Kexian Tang
user
KexianTang
67ef4be0985aa66b67021ddf
Hao Li
67ef4be0985aa66b67021de0
Zicheng Zhang
claimed_verified
2025-04-08T06:56:31.691Z
6526cc6bab4f5d98382f5603
https://cdn-avatars.huggingface.co/v1/production/uploads/6526cc6bab4f5d98382f5603/NekYYb61I5nt4au6gXsVK.jpeg
Zicheng Zhang
user
zhangzicheng
67ef4be0985aa66b67021de1
Guangtao Zhai
admin_assigned
2025-04-04T07:22:23.057Z
65535b125413c1a54e6fb243
/avatars/03bcf1d58865f5406aff49a415e78bdc.svg
Guangtao Zhai
user
GTZhai
67ef4be0985aa66b67021de2
Junchi Yan
admin_assigned
2025-04-04T07:22:30.311Z
667289f903c802764985d8c6
/avatars/916befcbf0e52ce56be49617f31c7bb2.svg
Junchi Yan
user
Rethinker
67ef4be0985aa66b67021de3
Hua Yang
67ef4be0985aa66b67021de4
Xue Yang
claimed_verified
2025-04-04T07:09:09.267Z
648e77184cae4f6921dbb382
https://cdn-avatars.huggingface.co/v1/production/uploads/648e77184cae4f6921dbb382/zAAJRvOStC0wZplqVWrk_.jpeg
Xue Yang
user
yangxue
67ef4be0985aa66b67021de5
Haodong Duan
admin_assigned
2025-04-04T07:22:37.146Z
63ee1379190ddd6214efd73a
https://cdn-avatars.huggingface.co/v1/production/uploads/1676546883247-noauth.png
HAODONG DUAN
user
KennyUTC
Envisioning Beyond the Pixels: Benchmarking Reasoning-Informed Visual
  Editing
Large Multi-modality Models (LMMs) have made significant progress in visual
understanding and generation, but they still face challenges in General Visual
Editing, particularly in following complex instructions, preserving appearance
consistency, and supporting flexible input formats. To address this gap, we
introduce RISEBench, the first benchmark for evaluating Reasoning-Informed
viSual Editing (RISE). RISEBench focuses on four key reasoning types: Temporal,
Causal, Spatial, and Logical Reasoning. We curate high-quality test cases for
each category and propose an evaluation framework that assesses Instruction
Reasoning, Appearance Consistency, and Visual Plausibility with both human
judges and an LMM-as-a-judge approach. Our experiments reveal that while
GPT-4o-Native significantly outperforms other open-source and proprietary
models, even this state-of-the-art system struggles with logical reasoning
tasks, highlighting an area that remains underexplored. As an initial effort,
RISEBench aims to provide foundational insights into reasoning-aware visual
editing and to catalyze future research. Though still in its early stages, we
are committed to continuously expanding and refining the benchmark to support
more comprehensive, reliable, and scalable evaluations of next-generation
multimodal systems. Our code and data will be released at
https://github.com/PhoenixZ810/RISEBench.
RISEBench evaluates Reasoning-Informed Visual Editing by assessing temporal, causal, spatial, and logical reasoning in multimodal models like GPT-4o-Native.
Reasoning-Informed viSual Editing Temporal Causal Spatial Logical Reasoning Instruction Reasoning Appearance Consistency Visual Plausibility LMM-as-a-judge multimodal models GPT-4o-Native