VLA-RFT: Vision-Language-Action Reinforcement Fine-tuning with Verified
  Rewards in World Simulators
Vision-Language-Action (VLA) models enable embodied decision-making but rely
heavily on imitation learning, leading to compounding errors and poor
robustness under distribution shift. Reinforcement learning (RL) can mitigate
these issues yet typically demands costly real-world interactions or suffers
from sim-to-real gaps. We introduce VLA-RFT, a reinforcement fine-tuning
framework that leverages a data-driven world model as a controllable simulator.
Trained from real interaction data, the simulator predicts future visual
observations conditioned on actions, allowing policy rollouts with dense,
trajectory-level rewards derived from goal-achieving references. This design
delivers an efficient and action-aligned learning signal, drastically lowering
sample requirements. With fewer than 400 fine-tuning steps, VLA-RFT surpasses
strong supervised baselines and achieves greater efficiency than
simulator-based RL. Moreover, it exhibits strong robustness under perturbed
conditions, sustaining stable task execution. Our results establish
world-model-based RFT as a practical post-training paradigm to enhance the
generalization and robustness of VLA models. For more details, please refer to
https://vla-rft.github.io/.
68dddcb66024653e8a3ed095
Hengtao Li
68dddcb66024653e8a3ed096
Pengxiang Ding
68dddcb66024653e8a3ed097
Runze Suo
68dddcb66024653e8a3ed098
Yihao Wang
claimed_verified
2025-10-02T13:55:32.098Z
68c429ef60f075cc46cc9cff
/avatars/21d277a1b46eabc967121a111e270cdd.svg
yh-wang
user
yh-wang
68dddcb66024653e8a3ed099
Zirui Ge
68dddcb66024653e8a3ed09a
Dongyuan Zang
68dddcb66024653e8a3ed09b
Kexian Yu
68dddcb66024653e8a3ed09c
Mingyang Sun
68dddcb66024653e8a3ed09d
Hongyin Zhang
68dddcb66024653e8a3ed09e
Donglin Wang
68dddcb66024653e8a3ed09f
Weihua Su
VLA-RFT: Vision-Language-Action Reinforcement Fine-tuning with Verified
  Rewards in World Simulators
Vision-Language-Action (VLA) models enable embodied decision-making but rely
heavily on imitation learning, leading to compounding errors and poor
robustness under distribution shift. Reinforcement learning (RL) can mitigate
these issues yet typically demands costly real-world interactions or suffers
from sim-to-real gaps. We introduce VLA-RFT, a reinforcement fine-tuning
framework that leverages a data-driven world model as a controllable simulator.
Trained from real interaction data, the simulator predicts future visual
observations conditioned on actions, allowing policy rollouts with dense,
trajectory-level rewards derived from goal-achieving references. This design
delivers an efficient and action-aligned learning signal, drastically lowering
sample requirements. With fewer than 400 fine-tuning steps, VLA-RFT surpasses
strong supervised baselines and achieves greater efficiency than
simulator-based RL. Moreover, it exhibits strong robustness under perturbed
conditions, sustaining stable task execution. Our results establish
world-model-based RFT as a practical post-training paradigm to enhance the
generalization and robustness of VLA models. For more details, please refer to
https://vla-rft.github.io/.
VLA-RFT uses a data-driven world model to fine-tune VLA models efficiently, reducing sample requirements and improving robustness under perturbations.
reinforcement learning imitation learning distribution shift world model policy rollouts trajectory-level rewards goal-achieving references sample requirements generalization robustness