MMBench-GUI: Hierarchical Multi-Platform Evaluation Framework for GUI
  Agents
We introduce MMBench-GUI, a hierarchical benchmark for evaluating GUI
automation agents across Windows, macOS, Linux, iOS, Android, and Web
platforms. It comprises four levels: GUI Content Understanding, Element
Grounding, Task Automation, and Task Collaboration, covering essential skills
for GUI agents. In addition, we propose a novel Efficiency-Quality Area (EQA)
metric to assess GUI agent execution efficiency in online automation scenarios.
Through MMBench-GUI, we identify accurate visual grounding as a critical
determinant of overall task success, emphasizing the substantial benefits of
modular frameworks that integrate specialized grounding modules. Furthermore,
to achieve reliable GUI automation, an agent requires strong task planning and
cross-platform generalization abilities, with long-context memory, a broad
action space, and long-term reasoning playing a critical role. More important,
task efficiency remains a critically underexplored dimension, and all models
suffer from substantial inefficiencies, with excessive redundant steps even
when tasks are ultimately completed. The integration of precise localization,
effective planning, and early stopping strategies is indispensable to enable
truly efficient and scalable GUI automation. Our benchmark code, evaluation
data, and running environment will be publicly available at
https://github.com/open-compass/MMBench-GUI.
68877a6bc8db48c925156eec
Xuehui Wang
68877a6bc8db48c925156eed
Zhenyu Wu
68877a6bc8db48c925156eee
JingJing Xie
claimed_verified
2025-07-29T07:10:00.031Z
6502f241b1792803da7e8def
https://cdn-avatars.huggingface.co/v1/production/uploads/6502f241b1792803da7e8def/mJ1XCVKivsMLi2Lo1kGKX.png
JingJing Xie
user
ownerEli
68877a6bc8db48c925156eef
Zichen Ding
claimed_verified
2025-07-29T07:09:51.687Z
642b9861bb77f8456634b048
https://cdn-avatars.huggingface.co/v1/production/uploads/642b9861bb77f8456634b048/VrNmmcdgX7FufQmdP5YaG.jpeg
Zichen Ding
user
heroding77
68877a6bc8db48c925156ef0
Bowen Yang
claimed_verified
2025-09-21T13:13:44.133Z
66abb1d1930106b4b433f295
/avatars/12e3375fb6aa04fc24d5092bf40cdecd.svg
ybw
user
YYangzzzz
68877a6bc8db48c925156ef1
Zehao Li
68877a6bc8db48c925156ef2
Zhaoyang Liu
claimed_verified
2025-09-19T06:51:17.452Z
6432c6152bfb2b0ec7572479
/avatars/7c2f908f511cdfa38a19e967d86f9c40.svg
Zhaoyang Liu
user
zyliu
68877a6bc8db48c925156ef3
Qingyun Li
claimed_verified
2025-07-29T07:09:53.977Z
637f100c1dbae09191088e4c
https://cdn-avatars.huggingface.co/v1/production/uploads/1669271529782-noauth.jpeg
Qingyun Li
user
Qingyun
68877a6bc8db48c925156ef4
Xuan Dong
68877a6bc8db48c925156ef5
Zhe Chen
68877a6bc8db48c925156ef6
Weiyun Wang
claimed_verified
2025-08-25T08:28:58.879Z
619507e7b74b6c591f794340
https://cdn-avatars.huggingface.co/v1/production/uploads/619507e7b74b6c591f794340/JbPDoy6Ko1V1-6oJJwFV8.jpeg
Weiyun Wang
user
Weiyun1025
68877a6bc8db48c925156ef7
Xiangyu Zhao
68877a6bc8db48c925156ef8
Jixuan Chen
68877a6bc8db48c925156ef9
Haodong Duan
claimed_verified
2025-08-27T07:11:59.649Z
63ee1379190ddd6214efd73a
https://cdn-avatars.huggingface.co/v1/production/uploads/1676546883247-noauth.png
HAODONG DUAN
user
KennyUTC
68877a6bc8db48c925156efa
Tianbao Xie
claimed_verified
2025-08-14T13:40:28.941Z
618767e4238063b4615d042b
https://cdn-avatars.huggingface.co/v1/production/uploads/1636263880877-noauth.jpeg
Tianbao Xie
user
tianbaoxiexxx
68877a6bc8db48c925156efb
Chenyu Yang
claimed_verified
2025-07-29T07:09:56.490Z
637f347a52229c639211bee8
https://cdn-avatars.huggingface.co/v1/production/uploads/637f347a52229c639211bee8/I9_PET-_6SJQJ6hXrACV4.jpeg
Chenyu Yang
user
cyyang822
68877a6bc8db48c925156efc
Shiqian Su
68877a6bc8db48c925156efd
Yue Yu
68877a6bc8db48c925156efe
Yuan Huang
68877a6bc8db48c925156eff
Yiqian Liu
68877a6bc8db48c925156f00
Xiao Zhang
68877a6bc8db48c925156f01
Yanting Zhang
68877a6bc8db48c925156f02
Xiangyu Yue
68877a6bc8db48c925156f03
Weijie Su
68877a6bc8db48c925156f04
Xizhou Zhu
68877a6bc8db48c925156f05
Wei Shen
68877a6bc8db48c925156f06
Jifeng Dai
68877a6bc8db48c925156f07
Wenhai Wang
MMBench-GUI: Hierarchical Multi-Platform Evaluation Framework for GUI
  Agents
We introduce MMBench-GUI, a hierarchical benchmark for evaluating GUI
automation agents across Windows, macOS, Linux, iOS, Android, and Web
platforms. It comprises four levels: GUI Content Understanding, Element
Grounding, Task Automation, and Task Collaboration, covering essential skills
for GUI agents. In addition, we propose a novel Efficiency-Quality Area (EQA)
metric to assess GUI agent execution efficiency in online automation scenarios.
Through MMBench-GUI, we identify accurate visual grounding as a critical
determinant of overall task success, emphasizing the substantial benefits of
modular frameworks that integrate specialized grounding modules. Furthermore,
to achieve reliable GUI automation, an agent requires strong task planning and
cross-platform generalization abilities, with long-context memory, a broad
action space, and long-term reasoning playing a critical role. More important,
task efficiency remains a critically underexplored dimension, and all models
suffer from substantial inefficiencies, with excessive redundant steps even
when tasks are ultimately completed. The integration of precise localization,
effective planning, and early stopping strategies is indispensable to enable
truly efficient and scalable GUI automation. Our benchmark code, evaluation
data, and running environment will be publicly available at
https://github.com/open-compass/MMBench-GUI.
MMBench-GUI evaluates GUI automation agents across multiple platforms using a hierarchical benchmark and Efficiency-Quality Area metric, highlighting the importance of visual grounding, task planning, and efficiency.
GUI Content Understanding Element Grounding Task Automation Task Collaboration Efficiency-Quality Area (EQA) visual grounding modular frameworks task planning cross-platform generalization long-context memory long-term reasoning task efficiency precise localization effective planning early stopping strategies