UI-Venus Technical Report: Building High-performance UI Agents with RFT
We present UI-Venus, a native UI agent that takes only screenshots as input
based on a multimodal large language model. UI-Venus achieves SOTA performance
on both UI grounding and navigation tasks using only several hundred thousand
high-quality training samples through reinforcement finetune (RFT) based on
Qwen2.5-VL. Specifically, the 7B and 72B variants of UI-Venus obtain 94.1% /
50.8% and 95.3% / 61.9% on the standard grounding benchmarks, i.e.,
Screenspot-V2 / Pro, surpassing the previous SOTA baselines including
open-source GTA1 and closed-source UI-TARS-1.5.To show UI-Venus's summary and
planing ability, we also evaluate it on the AndroidWorld, an online UI
navigation arena, on which our 7B and 72B variants achieve 49.1% and 65.9%
success rate, also beating existing models.To achieve this, we introduce
carefully designed reward functions for both UI grounding and navigation tasks
and corresponding efficient data cleaning strategies.To further boost
navigation performance, we propose Self-Evolving Trajectory History Alignment
\& Sparse Action Enhancement that refine historical reasoning traces and
balances the distribution of sparse but critical actions, leading to more
coherent planning and better generalization in complex UI tasks. Our
contributions include the publish of SOTA open-source UI agents, comprehensive
data cleaning protocols and a novel self-evolving framework for improving
navigation performance, which encourage further research and development in the
community. Code is available at https://github.com/antgroup/UI-Venus.
689e97cda4caabb4320e5ce7
Zhangxuan Gu
claimed_verified
2025-08-18T06:57:34.206Z
60d2a2984956988b63753371
https://cdn-avatars.huggingface.co/v1/production/uploads/60d2a2984956988b63753371/apXIcWbi7jnLVH37CdMTV.jpeg
Zhangxuan Gu
user
zhangxgu
689e97cda4caabb4320e5ce8
Zhengwen Zeng
claimed_verified
2025-08-19T10:49:56.579Z
64cb238576200ec80fe988f8
/avatars/42c48710c7881c9dfbcc075fec3cb600.svg
zeus
user
zengw
689e97cda4caabb4320e5ce9
Zhenyu Xu
689e97cda4caabb4320e5cea
Xingran Zhou
689e97cda4caabb4320e5ceb
Shuheng Shen
689e97cda4caabb4320e5cec
Yunfei Liu
689e97cda4caabb4320e5ced
Beitong Zhou
689e97cda4caabb4320e5cee
Changhua Meng
689e97cda4caabb4320e5cef
Tianyu Xia
689e97cda4caabb4320e5cf0
Weizhi Chen
689e97cda4caabb4320e5cf1
Yue Wen
689e97cda4caabb4320e5cf2
Jingya Dou
689e97cda4caabb4320e5cf3
Fei Tang
689e97cda4caabb4320e5cf4
Jinzhen Lin
689e97cda4caabb4320e5cf5
Yulin Liu
689e97cda4caabb4320e5cf6
Zhenlin Guo
689e97cda4caabb4320e5cf7
Yichen Gong
689e97cda4caabb4320e5cf8
Heng Jia
689e97cda4caabb4320e5cf9
Changlong Gao
689e97cda4caabb4320e5cfa
Yuan Guo
689e97cda4caabb4320e5cfb
Yong Deng
689e97cda4caabb4320e5cfc
Zhenyu Guo
689e97cda4caabb4320e5cfd
Liang Chen
689e97cda4caabb4320e5cfe
Weiqiang Wang
UI-Venus Technical Report: Building High-performance UI Agents with RFT
We present UI-Venus, a native UI agent that takes only screenshots as input
based on a multimodal large language model. UI-Venus achieves SOTA performance
on both UI grounding and navigation tasks using only several hundred thousand
high-quality training samples through reinforcement finetune (RFT) based on
Qwen2.5-VL. Specifically, the 7B and 72B variants of UI-Venus obtain 94.1% /
50.8% and 95.3% / 61.9% on the standard grounding benchmarks, i.e.,
Screenspot-V2 / Pro, surpassing the previous SOTA baselines including
open-source GTA1 and closed-source UI-TARS-1.5.To show UI-Venus's summary and
planing ability, we also evaluate it on the AndroidWorld, an online UI
navigation arena, on which our 7B and 72B variants achieve 49.1% and 65.9%
success rate, also beating existing models.To achieve this, we introduce
carefully designed reward functions for both UI grounding and navigation tasks
and corresponding efficient data cleaning strategies.To further boost
navigation performance, we propose Self-Evolving Trajectory History Alignment
\& Sparse Action Enhancement that refine historical reasoning traces and
balances the distribution of sparse but critical actions, leading to more
coherent planning and better generalization in complex UI tasks. Our
contributions include the publish of SOTA open-source UI agents, comprehensive
data cleaning protocols and a novel self-evolving framework for improving
navigation performance, which encourage further research and development in the
community. Code is available at https://github.com/antgroup/UI-Venus.
UI-Venus, a multimodal large language model-based UI agent, achieves state-of-the-art performance in UI grounding and navigation tasks using reinforcement fine-tuning and novel self-evolving frameworks.
multimodal large language model reinforcement finetune Qwen2.5-VL UI grounding navigation tasks Screenspot-V2 Pro AndroidWorld reward functions data cleaning strategies Self-Evolving Trajectory History Alignment Sparse Action Enhancement