ERA: Transforming VLMs into Embodied Agents via Embodied Prior Learning
  and Online Reinforcement Learning
Recent advances in embodied AI highlight the potential of vision language
models (VLMs) as agents capable of perception, reasoning, and interaction in
complex environments. However, top-performing systems rely on large-scale
models that are costly to deploy, while smaller VLMs lack the necessary
knowledge and skills to succeed. To bridge this gap, we present
Embodied Reasoning Agent (ERA), a two-stage framework that integrates
prior knowledge learning and online reinforcement learning (RL). The first
stage, Embodied Prior Learning, distills foundational knowledge from
three types of data: (1) Trajectory-Augmented Priors, which enrich existing
trajectory data with structured reasoning generated by stronger models; (2)
Environment-Anchored Priors, which provide in-environment knowledge and
grounding supervision; and (3) External Knowledge Priors, which transfer
general knowledge from out-of-environment datasets. In the second stage, we
develop an online RL pipeline that builds on these priors to further enhance
agent performance. To overcome the inherent challenges in agent RL, including
long horizons, sparse rewards, and training instability, we introduce three key
designs: self-summarization for context management, dense reward shaping, and
turn-level policy optimization. Extensive experiments on both high-level
planning (EB-ALFRED) and low-level control (EB-Manipulation) tasks demonstrate
that ERA-3B surpasses both prompting-based large models and previous
training-based baselines. Specifically, it achieves overall improvements of
8.4\% on EB-ALFRED and 19.4\% on EB-Manipulation over GPT-4o, and exhibits
strong generalization to unseen tasks. Overall, ERA offers a practical path
toward scalable embodied intelligence, providing methodological insights for
future embodied AI systems.
68ef0007486b78128f0e339c
Hanyang Chen
claimed_verified
2025-10-15T07:07:38.765Z
6700b1f93381f2db06857fb5
/avatars/c8b9ec7c00773c5a4055ba50de0c6b2f.svg
Hanyang Chen
user
Hanyang81
68ef0007486b78128f0e339d
Mark Zhao
68ef0007486b78128f0e339e
Rui Yang
claimed_verified
2025-10-15T03:12:56.270Z
64d45451c34a346181b130dd
/avatars/9bb8205b889337df5d321539c9b5d69d.svg
Rui Yang
user
Ray2333
68ef0007486b78128f0e339f
Qinwei Ma
68ef0007486b78128f0e33a0
Ke Yang
68ef0007486b78128f0e33a1
Jiarui Yao
68ef0007486b78128f0e33a2
Kangrui Wang
68ef0007486b78128f0e33a3
Hao Bai
68ef0007486b78128f0e33a4
Zhenhailong Wang
68ef0007486b78128f0e33a5
Rui Pan
68ef0007486b78128f0e33a6
Mengchao Zhang
68ef0007486b78128f0e33a7
Jose Barreiros
68ef0007486b78128f0e33a8
Aykut Onol
68ef0007486b78128f0e33a9
ChengXiang Zhai
68ef0007486b78128f0e33aa
Heng Ji
68ef0007486b78128f0e33ab
Manling Li
68ef0007486b78128f0e33ac
Huan Zhang
68ef0007486b78128f0e33ad
Tong Zhang
ERA: Transforming VLMs into Embodied Agents via Embodied Prior Learning
  and Online Reinforcement Learning
Recent advances in embodied AI highlight the potential of vision language
models (VLMs) as agents capable of perception, reasoning, and interaction in
complex environments. However, top-performing systems rely on large-scale
models that are costly to deploy, while smaller VLMs lack the necessary
knowledge and skills to succeed. To bridge this gap, we present
Embodied Reasoning Agent (ERA), a two-stage framework that integrates
prior knowledge learning and online reinforcement learning (RL). The first
stage, Embodied Prior Learning, distills foundational knowledge from
three types of data: (1) Trajectory-Augmented Priors, which enrich existing
trajectory data with structured reasoning generated by stronger models; (2)
Environment-Anchored Priors, which provide in-environment knowledge and
grounding supervision; and (3) External Knowledge Priors, which transfer
general knowledge from out-of-environment datasets. In the second stage, we
develop an online RL pipeline that builds on these priors to further enhance
agent performance. To overcome the inherent challenges in agent RL, including
long horizons, sparse rewards, and training instability, we introduce three key
designs: self-summarization for context management, dense reward shaping, and
turn-level policy optimization. Extensive experiments on both high-level
planning (EB-ALFRED) and low-level control (EB-Manipulation) tasks demonstrate
that ERA-3B surpasses both prompting-based large models and previous
training-based baselines. Specifically, it achieves overall improvements of
8.4\% on EB-ALFRED and 19.4\% on EB-Manipulation over GPT-4o, and exhibits
strong generalization to unseen tasks. Overall, ERA offers a practical path
toward scalable embodied intelligence, providing methodological insights for
future embodied AI systems.