UI-R1: Enhancing Action Prediction of GUI Agents by Reinforcement
  Learning
The recent DeepSeek-R1 has showcased the emergence of reasoning capabilities
in LLMs through reinforcement learning (RL) with rule-based rewards. Building
on this idea, we are the first to explore how rule-based RL can enhance the
reasoning capabilities of multimodal large language models (MLLMs) for graphic
user interface (GUI) action prediction tasks. To this end, we curate a small
yet high-quality dataset of 136 challenging tasks, encompassing five common
action types on mobile devices. We also introduce a unified rule-based action
reward, enabling model optimization via policy-based algorithms such as Group
Relative Policy Optimization (GRPO). Experimental results demonstrate that our
proposed data-efficient model, UI-R1-3B, achieves substantial improvements on
both in-domain (ID) and out-of-domain (OOD) tasks. Specifically, on the ID
benchmark AndroidControl, the action type accuracy improves by 15%, while
grounding accuracy increases by 10.3%, compared with the base model (i.e.
Qwen2.5-VL-3B). On the OOD GUI grounding benchmark ScreenSpot-Pro, our model
surpasses the base model by 6.0% and achieves competitive performance with
larger models (e.g., OS-Atlas-7B), which are trained via supervised fine-tuning
(SFT) on 76K data. These results underscore the potential of rule-based
reinforcement learning to advance GUI understanding and control, paving the way
for future research in this domain.
67e606fb6c44ab0376a498a1
Zhengxi Lu
admin_assigned
2025-03-28T08:42:01.155Z
676127cf11b19ea602bb202a
/avatars/dfd802a24bd63e509728159ebb1769f6.svg
Zhengxi Lu
user
LZXzju
67e606fb6c44ab0376a498a2
Yuxiang Chai
claimed_verified
2025-03-28T08:37:39.852Z
6458ce236fa580137af5aa95
/avatars/db65a7332e375eb5daad5c1b076b1e3b.svg
Yuxiang Chai
user
Yuxiang007
67e606fb6c44ab0376a498a3
Yaxuan Guo
admin_assigned
2025-03-28T08:42:12.269Z
65c237220c57a7141888363e
/avatars/ce43c52f47d524c5b747523058946325.svg
guoyaxuan
user
guoyaxuan0106
67e606fb6c44ab0376a498a4
Xi Yin
67e606fb6c44ab0376a498a5
Liang Liu
67e606fb6c44ab0376a498a6
Hao Wang
67e606fb6c44ab0376a498a7
Guanjing Xiong
67e606fb6c44ab0376a498a8
Hongsheng Li
admin_assigned
2025-03-28T08:43:30.298Z
65c04e9c27a5fdca81abcbd9
/avatars/12a155683c824fa23da4a9e2bed4f64e.svg
Hongsheng LI
user
hsli-cuhk
UI-R1: Enhancing Action Prediction of GUI Agents by Reinforcement
  Learning
The recent DeepSeek-R1 has showcased the emergence of reasoning capabilities
in LLMs through reinforcement learning (RL) with rule-based rewards. Building
on this idea, we are the first to explore how rule-based RL can enhance the
reasoning capabilities of multimodal large language models (MLLMs) for graphic
user interface (GUI) action prediction tasks. To this end, we curate a small
yet high-quality dataset of 136 challenging tasks, encompassing five common
action types on mobile devices. We also introduce a unified rule-based action
reward, enabling model optimization via policy-based algorithms such as Group
Relative Policy Optimization (GRPO). Experimental results demonstrate that our
proposed data-efficient model, UI-R1-3B, achieves substantial improvements on
both in-domain (ID) and out-of-domain (OOD) tasks. Specifically, on the ID
benchmark AndroidControl, the action type accuracy improves by 15%, while
grounding accuracy increases by 10.3%, compared with the base model (i.e.
Qwen2.5-VL-3B). On the OOD GUI grounding benchmark ScreenSpot-Pro, our model
surpasses the base model by 6.0% and achieves competitive performance with
larger models (e.g., OS-Atlas-7B), which are trained via supervised fine-tuning
(SFT) on 76K data. These results underscore the potential of rule-based
reinforcement learning to advance GUI understanding and control, paving the way
for future research in this domain.
Rule-based reinforcement learning enhances multimodal large language models for GUI action prediction, improving data efficiency and performance.
reinforcement learning multimodal large language models GUI action prediction Group Relative Policy Optimization policy-based algorithms AndroidControl ScreenSpot-Pro supervised fine-tuning