MAS-Bench: A Unified Benchmark for Shortcut-Augmented Hybrid Mobile GUI
  Agents
To enhance the efficiency of GUI agents on various platforms like smartphones
and computers, a hybrid paradigm that combines flexible GUI operations with
efficient shortcuts (e.g., API, deep links) is emerging as a promising
direction. However, a framework for systematically benchmarking these hybrid
agents is still underexplored. To take the first step in bridging this gap, we
introduce MAS-Bench, a benchmark that pioneers the evaluation of GUI-shortcut
hybrid agents with a specific focus on the mobile domain. Beyond merely using
predefined shortcuts, MAS-Bench assesses an agent's capability to autonomously
generate shortcuts by discovering and creating reusable, low-cost workflows. It
features 139 complex tasks across 11 real-world applications, a knowledge base
of 88 predefined shortcuts (APIs, deep-links, RPA scripts), and 7 evaluation
metrics. The tasks are designed to be solvable via GUI-only operations, but can
be significantly accelerated by intelligently embedding shortcuts. Experiments
show that hybrid agents achieve significantly higher success rates and
efficiency than their GUI-only counterparts. This result also demonstrates the
effectiveness of our method for evaluating an agent's shortcut generation
capabilities. MAS-Bench fills a critical evaluation gap, providing a
foundational platform for future advancements in creating more efficient and
robust intelligent agents.
68bf993b207285de11b07bc9
Pengxiang Zhao
claimed_verified
2025-09-09T13:47:46.445Z
65a088f4300957620ba45c70
/avatars/56ed45e10d3455531979f30881b2d3f9.svg
pengxiang zhao
user
Pengxiangzhao
68bf993b207285de11b07bca
Guangyi Liu
claimed_verified
2025-09-09T13:47:49.267Z
64d761b98ebc40443831f82a
https://cdn-avatars.huggingface.co/v1/production/uploads/64d761b98ebc40443831f82a/DHBOtOstiFp2-lDY6b9gb.png
Guangyi Liu
user
lgy0404
68bf993b207285de11b07bcb
Yaozhen Liang
68bf993b207285de11b07bcc
Weiqing He
68bf993b207285de11b07bcd
Zhengxi Lu
claimed_verified
2025-09-16T09:43:33.294Z
676127cf11b19ea602bb202a
/avatars/dfd802a24bd63e509728159ebb1769f6.svg
Zhengxi Lu
user
LZXzju
68bf993b207285de11b07bce
Yuehao Huang
68bf993b207285de11b07bcf
Yaxuan Guo
68bf993b207285de11b07bd0
Kexin Zhang
68bf993b207285de11b07bd1
Hao Wang
68bf993b207285de11b07bd2
Liang Liu
68bf993b207285de11b07bd3
Yong Liu
MAS-Bench: A Unified Benchmark for Shortcut-Augmented Hybrid Mobile GUI
  Agents
To enhance the efficiency of GUI agents on various platforms like smartphones
and computers, a hybrid paradigm that combines flexible GUI operations with
efficient shortcuts (e.g., API, deep links) is emerging as a promising
direction. However, a framework for systematically benchmarking these hybrid
agents is still underexplored. To take the first step in bridging this gap, we
introduce MAS-Bench, a benchmark that pioneers the evaluation of GUI-shortcut
hybrid agents with a specific focus on the mobile domain. Beyond merely using
predefined shortcuts, MAS-Bench assesses an agent's capability to autonomously
generate shortcuts by discovering and creating reusable, low-cost workflows. It
features 139 complex tasks across 11 real-world applications, a knowledge base
of 88 predefined shortcuts (APIs, deep-links, RPA scripts), and 7 evaluation
metrics. The tasks are designed to be solvable via GUI-only operations, but can
be significantly accelerated by intelligently embedding shortcuts. Experiments
show that hybrid agents achieve significantly higher success rates and
efficiency than their GUI-only counterparts. This result also demonstrates the
effectiveness of our method for evaluating an agent's shortcut generation
capabilities. MAS-Bench fills a critical evaluation gap, providing a
foundational platform for future advancements in creating more efficient and
robust intelligent agents.
MAS-Bench evaluates GUI-shortcut hybrid agents on mobile devices, demonstrating their superior performance and efficiency over GUI-only agents through a comprehensive benchmarking framework.
GUI agents hybrid paradigm efficient shortcuts API deep links MAS-Bench autonomous generation reusable workflows evaluation metrics success rates efficiency