SimpleVLA-RL: Scaling VLA Training via Reinforcement Learning
Vision-Language-Action (VLA) models have recently emerged as a powerful
paradigm for robotic manipulation. Despite substantial progress enabled by
large-scale pretraining and supervised fine-tuning (SFT), these models face two
fundamental challenges: (i) the scarcity and high cost of large-scale
human-operated robotic trajectories required for SFT scaling, and (ii) limited
generalization to tasks involving distribution shift. Recent breakthroughs in
Large Reasoning Models (LRMs) demonstrate that reinforcement learning (RL) can
dramatically enhance step-by-step reasoning capabilities, raising a natural
question: Can RL similarly improve the long-horizon step-by-step action
planning of VLA? In this work, we introduce SimpleVLA-RL, an efficient RL
framework tailored for VLA models. Building upon veRL, we introduce
VLA-specific trajectory sampling, scalable parallelization, multi-environment
rendering, and optimized loss computation. When applied to OpenVLA-OFT,
SimpleVLA-RL achieves SoTA performance on LIBERO and even outperforms pi_0
on RoboTwin 1.0\&2.0 with the exploration-enhancing strategies we introduce.
SimpleVLA-RL not only reduces dependence on large-scale data and enables robust
generalization, but also remarkably surpasses SFT in real-world tasks.
Moreover, we identify a novel phenomenon ``pushcut'' during RL training,
wherein the policy discovers previously unseen patterns beyond those seen in
the previous training process. Github: https://github.com/PRIME-RL/SimpleVLA-RL
68c37bb3fc1747b912403994
Haozhan Li
claimed_verified
2025-09-12T16:09:17.957Z
662f638ba9891e43cc4c5125
/avatars/77c22de5511f9b85d98ec75fb0b5e9be.svg
Li Haozhan
user
Haozhan72
68c37bb3fc1747b912403995
Yuxin Zuo
claimed_verified
2025-09-12T16:09:12.723Z
622474f38dc6b0b64f5e903d
/avatars/d6b60a014277a8ec7d564163c5f644aa.svg
Yuxin Zuo
user
yuxinzuo
68c37bb3fc1747b912403996
Jiale Yu
68c37bb3fc1747b912403997
Yuhao Zhang
claimed_verified
2025-09-12T16:09:05.320Z
68c3a50c2c2ff7b9c964ea70
/avatars/d122078c96a4961291ba271c2d982d8b.svg
Zhaohui Yang
user
Zhaohui2001aa
68c37bb3fc1747b912403998
Zhaohui Yang
68c37bb3fc1747b912403999
Kaiyan Zhang
claimed_verified
2025-09-12T16:08:51.461Z
60bc94cd85a3ab33829b6211
/avatars/b57d36c7577fbbb42ea5b963eef4144a.svg
Kaiyan Zhang
user
iseesaw
68c37bb3fc1747b91240399a
Xuekai Zhu
claimed_verified
2025-09-19T06:50:35.408Z
647ffddeb82adfa7cc1a10d9
/avatars/26aa168d6b2068298ebb16584aa52b6c.svg
zhu
user
xuekai
68c37bb3fc1747b91240399b
Yuchen Zhang
claimed_verified
2025-09-12T16:09:08.894Z
66e3f8fb5d97b5bb46923444
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/DW806I00-00oQAYvD4ocQ.png
Yuchen Zhang
user
YucZhang2003
68c37bb3fc1747b91240399c
Tianxing Chen
68c37bb3fc1747b91240399d
Ganqu Cui
68c37bb3fc1747b91240399e
Dehui Wang
claimed_verified
2025-09-12T16:08:48.434Z
668e3a4acbb3c5e08b6e4b71
/avatars/03e6edc5c5fe9074c3de3155125526db.svg
dehui wang
user
dehuiwang
68c37bb3fc1747b91240399f
Dingxiang Luo
68c37bb3fc1747b9124039a0
Yuchen Fan
68c37bb3fc1747b9124039a1
Youbang Sun
claimed_verified
2025-09-12T16:09:01.441Z
679ce8c048ebd7903d76a832
/avatars/5f3fecaacfee6e2d5a72dd19fe87055a.svg
Youbang Sun
user
Youbang
68c37bb3fc1747b9124039a2
Jia Zeng
68c37bb3fc1747b9124039a3
Jiangmiao Pang
68c37bb3fc1747b9124039a4
Shanghang Zhang
68c37bb3fc1747b9124039a5
Yu Wang
68c37bb3fc1747b9124039a6
Yao Mu
68c37bb3fc1747b9124039a7
Bowen Zhou
68c37bb3fc1747b9124039a8
Ning Ding
claimed_verified
2025-09-13T14:56:26.665Z
60cf4bcb1ce3775ebb86e5d5
/avatars/12bcd18d215abf91f297f93007733148.svg
Ning Ding
user
stingning
SimpleVLA-RL: Scaling VLA Training via Reinforcement Learning
Vision-Language-Action (VLA) models have recently emerged as a powerful
paradigm for robotic manipulation. Despite substantial progress enabled by
large-scale pretraining and supervised fine-tuning (SFT), these models face two
fundamental challenges: (i) the scarcity and high cost of large-scale
human-operated robotic trajectories required for SFT scaling, and (ii) limited
generalization to tasks involving distribution shift. Recent breakthroughs in
Large Reasoning Models (LRMs) demonstrate that reinforcement learning (RL) can
dramatically enhance step-by-step reasoning capabilities, raising a natural
question: Can RL similarly improve the long-horizon step-by-step action
planning of VLA? In this work, we introduce SimpleVLA-RL, an efficient RL
framework tailored for VLA models. Building upon veRL, we introduce
VLA-specific trajectory sampling, scalable parallelization, multi-environment
rendering, and optimized loss computation. When applied to OpenVLA-OFT,
SimpleVLA-RL achieves SoTA performance on LIBERO and even outperforms pi_0
on RoboTwin 1.0\&2.0 with the exploration-enhancing strategies we introduce.
SimpleVLA-RL not only reduces dependence on large-scale data and enables robust
generalization, but also remarkably surpasses SFT in real-world tasks.
Moreover, we identify a novel phenomenon ``pushcut'' during RL training,
wherein the policy discovers previously unseen patterns beyond those seen in
the previous training process. Github: https://github.com/PRIME-RL/SimpleVLA-RL
SimpleVLA-RL, an RL framework for VLA models, enhances long-horizon action planning, achieves state-of-the-art performance, and discovers novel patterns during training.
reinforcement learning RL VLA models trajectory sampling parallelization multi-environment rendering loss computation LIBERO RoboTwin step-by-step reasoning distribution shift pushcut