Efficient Multi-turn RL for GUI Agents via Decoupled Training and
  Adaptive Data Curation
Vision-language model (VLM) based GUI agents show promise for automating
complex desktop and mobile tasks, but face significant challenges in applying
reinforcement learning (RL): (1) slow multi-turn interactions with GUI
environments for policy rollout, and (2) insufficient high-quality
agent-environment interactions for policy learning. To address these
challenges, we propose DART, a Decoupled Agentic RL Training framework for GUI
agents, which coordinates heterogeneous modules in a highly decoupled manner.
DART separates the training system into four asynchronous modules: environment
cluster, rollout service, data manager, and trainer. This design enables
non-blocking communication, asynchronous training, rollout-wise trajectory
sampling, and per-worker model synchronization, significantly improving the
system efficiency: 1.6*GPU utilization for rollout, 1.9* training throughput,
and 5.5* environment utilization. To facilitate effective learning from
abundant samples, we introduce an adaptive data curation scheme: (1)
pre-collecting successful trajectories for challenging tasks to supplement
sparse success in online sampling; (2) dynamically adjusting rollout numbers
and trajectory lengths based on task difficulty; (3) training selectively on
high-entropy steps to prioritize critical decisions; (4) stabilizing learning
via truncated importance sampling for policy mismatch between policy rollout
and updating. On the OSWorld benchmark, DART-GUI-7B achieves a 42.13% task
success rate, a 14.61% absolute gain over the base model, and 7.34% higher than
open-source SOTA. We will fully open-source our training framework, data, and
model checkpoints via computer-use-agents.github.io/dart-gui, which we believe
is a timely contribution to the open-source community of agentic RL training.
68db503dd2bf1f4b15ec74b2
Pengxiang Li
claimed_verified
2025-10-02T03:09:11.570Z
65d6bd8ebb862d1b367580fe
https://cdn-avatars.huggingface.co/v1/production/uploads/65d6bd8ebb862d1b367580fe/Pf5o-Oxi2IZxJPa-ZgEtV.jpeg
Pengxiang Li
user
PengxiangLi
68db503dd2bf1f4b15ec74b3
Zechen Hu
68db503dd2bf1f4b15ec74b4
Zirui Shang
68db503dd2bf1f4b15ec74b5
Jingrong Wu
68db503dd2bf1f4b15ec74b6
Yang Liu
68db503dd2bf1f4b15ec74b7
Hui Liu
68db503dd2bf1f4b15ec74b8
Zhi Gao
68db503dd2bf1f4b15ec74b9
Chenrui Shi
68db503dd2bf1f4b15ec74ba
Bofei Zhang
claimed_verified
2025-10-09T06:52:25.405Z
643e2ff70efe9e9de09c3881
/avatars/8e4433332bf6983e1b7956d0c96241b2.svg
Bofei Zhang
user
Bofeee5675
68db503dd2bf1f4b15ec74bb
Zihao Zhang
68db503dd2bf1f4b15ec74bc
Xiaochuan Shi
68db503dd2bf1f4b15ec74bd
Zedong YU
68db503dd2bf1f4b15ec74be
Yuwei Wu
68db503dd2bf1f4b15ec74bf
Xinxiao Wu
68db503dd2bf1f4b15ec74c0
Yunde Jia
68db503dd2bf1f4b15ec74c1
Liuyu Xiang
68db503dd2bf1f4b15ec74c2
Zhaofeng He
68db503dd2bf1f4b15ec74c3
Qing Li
Efficient Multi-turn RL for GUI Agents via Decoupled Training and
  Adaptive Data Curation
Vision-language model (VLM) based GUI agents show promise for automating
complex desktop and mobile tasks, but face significant challenges in applying
reinforcement learning (RL): (1) slow multi-turn interactions with GUI
environments for policy rollout, and (2) insufficient high-quality
agent-environment interactions for policy learning. To address these
challenges, we propose DART, a Decoupled Agentic RL Training framework for GUI
agents, which coordinates heterogeneous modules in a highly decoupled manner.
DART separates the training system into four asynchronous modules: environment
cluster, rollout service, data manager, and trainer. This design enables
non-blocking communication, asynchronous training, rollout-wise trajectory
sampling, and per-worker model synchronization, significantly improving the
system efficiency: 1.6*GPU utilization for rollout, 1.9* training throughput,
and 5.5* environment utilization. To facilitate effective learning from
abundant samples, we introduce an adaptive data curation scheme: (1)
pre-collecting successful trajectories for challenging tasks to supplement
sparse success in online sampling; (2) dynamically adjusting rollout numbers
and trajectory lengths based on task difficulty; (3) training selectively on
high-entropy steps to prioritize critical decisions; (4) stabilizing learning
via truncated importance sampling for policy mismatch between policy rollout
and updating. On the OSWorld benchmark, DART-GUI-7B achieves a 42.13% task
success rate, a 14.61% absolute gain over the base model, and 7.34% higher than
open-source SOTA. We will fully open-source our training framework, data, and
model checkpoints via computer-use-agents.github.io/dart-gui, which we believe
is a timely contribution to the open-source community of agentic RL training.
DART, a decoupled reinforcement learning framework for GUI agents, improves efficiency and learning effectiveness through asynchronous modules and adaptive data curation, achieving high task success rates on the OSWorld benchmark.
reinforcement learning RL GUI agents DART Decoupled Agentic RL Training environment cluster rollout service data manager trainer non-blocking communication asynchronous training rollout-wise trajectory sampling per-worker model synchronization adaptive data curation successful trajectories rollout numbers trajectory lengths high-entropy steps truncated importance sampling OSWorld benchmark DART-GUI-7B