MobA: A Two-Level Agent System for Efficient Mobile Task Automation
Current mobile assistants are limited by dependence on system APIs or
struggle with complex user instructions and diverse interfaces due to
restricted comprehension and decision-making abilities. To address these
challenges, we propose MobA, a novel Mobile phone Agent powered by multimodal
large language models that enhances comprehension and planning capabilities
through a sophisticated two-level agent architecture. The high-level Global
Agent (GA) is responsible for understanding user commands, tracking history
memories, and planning tasks. The low-level Local Agent (LA) predicts detailed
actions in the form of function calls, guided by sub-tasks and memory from the
GA. Integrating a Reflection Module allows for efficient task completion and
enables the system to handle previously unseen complex tasks. MobA demonstrates
significant improvements in task execution efficiency and completion rate in
real-life evaluations, underscoring the potential of MLLM-empowered mobile
assistants.
6711e2dd1133b5e4de846c00
Zichen Zhu
claimed_verified
2024-10-18T10:32:58.239Z
64d4b50a8ebc404438d86961
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/dxdusVp085Lx9hZ-y9fMp.jpeg
Zichen Zhu
user
JamesZhutheThird
6711e2dd1133b5e4de846c01
Hao Tang
6711e2dd1133b5e4de846c02
Yansi Li
6711e2dd1133b5e4de846c03
Kunyao Lan
admin_assigned
2024-10-18T12:34:20.631Z
62a430b7f4ec0e5e8760c776
/avatars/708f7558ceeffdc41bb9c871c8f09409.svg
Kunyao Lan
user
lankunyao
6711e2dd1133b5e4de846c04
Yixuan Jiang
6711e2dd1133b5e4de846c05
Hao Zhou
6711e2dd1133b5e4de846c06
Yixiao Wang
6711e2dd1133b5e4de846c07
Situo Zhang
admin_assigned
2024-10-18T12:35:16.260Z
64a6217e063f497473d7cab8
/avatars/45e1a7ed953353a3602924374d2ac163.svg
Situo Zhang
user
Situo
6711e2dd1133b5e4de846c08
Liangtai Sun
6711e2dd1133b5e4de846c09
Lu Chen
6711e2dd1133b5e4de846c0a
Kai Yu
MobA: A Two-Level Agent System for Efficient Mobile Task Automation
Current mobile assistants are limited by dependence on system APIs or
struggle with complex user instructions and diverse interfaces due to
restricted comprehension and decision-making abilities. To address these
challenges, we propose MobA, a novel Mobile phone Agent powered by multimodal
large language models that enhances comprehension and planning capabilities
through a sophisticated two-level agent architecture. The high-level Global
Agent (GA) is responsible for understanding user commands, tracking history
memories, and planning tasks. The low-level Local Agent (LA) predicts detailed
actions in the form of function calls, guided by sub-tasks and memory from the
GA. Integrating a Reflection Module allows for efficient task completion and
enables the system to handle previously unseen complex tasks. MobA demonstrates
significant improvements in task execution efficiency and completion rate in
real-life evaluations, underscoring the potential of MLLM-empowered mobile
assistants.
MobA, a mobile assistant using multimodal large language models, improves task execution through a dual-agent architecture with reflection, handling complex instructions effectively.
multimodal large language models high-level Global Agent low-level Local Agent Reflection Module