Being-0: A Humanoid Robotic Agent with Vision-Language Models and
  Modular Skills
Building autonomous robotic agents capable of achieving human-level
performance in real-world embodied tasks is an ultimate goal in humanoid robot
research. Recent advances have made significant progress in high-level
cognition with Foundation Models (FMs) and low-level skill development for
humanoid robots. However, directly combining these components often results in
poor robustness and efficiency due to compounding errors in long-horizon tasks
and the varied latency of different modules. We introduce Being-0, a
hierarchical agent framework that integrates an FM with a modular skill
library. The FM handles high-level cognitive tasks such as instruction
understanding, task planning, and reasoning, while the skill library provides
stable locomotion and dexterous manipulation for low-level control. To bridge
the gap between these levels, we propose a novel Connector module, powered by a
lightweight vision-language model (VLM). The Connector enhances the FM's
embodied capabilities by translating language-based plans into actionable skill
commands and dynamically coordinating locomotion and manipulation to improve
task success. With all components, except the FM, deployable on low-cost
onboard computation devices, Being-0 achieves efficient, real-time performance
on a full-sized humanoid robot equipped with dexterous hands and active vision.
Extensive experiments in large indoor environments demonstrate Being-0's
effectiveness in solving complex, long-horizon tasks that require challenging
navigation and manipulation subtasks. For further details and videos, visit
https://beingbeyond.github.io/being-0.
67d8eadc045f869fea1ce3f2
Haoqi Yuan
claimed_verified
2025-03-18T11:33:52.255Z
644560657a7b94ddc2d445a3
/avatars/09d6447da6ff1bd0b2b00c899c9f1b28.svg
Haoqi Yuan
user
Yaya041
67d8eadc045f869fea1ce3f3
Yu Bai
claimed_verified
2025-05-27T07:56:14.250Z
64b964d31043081735692158
/avatars/209ae30594e39f9fbfa2972689b1a8b1.svg
Yu Bai
user
woodenwhite
67d8eadc045f869fea1ce3f4
Yuhui Fu
admin_assigned
2025-03-18T09:17:46.443Z
67d92b2218de6ef86c60f7d4
/avatars/9758522c99bc38bc7b60845eff8bf8d7.svg
Yuhui Fu
user
fuyh
67d8eadc045f869fea1ce3f5
Bohan Zhou
67d8eadc045f869fea1ce3f6
Yicheng Feng
admin_assigned
2025-03-18T09:17:31.771Z
6655b86e607894ea80d74910
/avatars/663c0135c903c9c127fe1b8d8aaf279c.svg
yicheng feng
user
takenpeanut
67d8eadc045f869fea1ce3f7
Xinrun Xu
admin_assigned
2025-03-18T09:17:05.200Z
653238fdcd5377e9adee0c41
/avatars/78aea70cde6ab0050c7e18b5e148075c.svg
Xinrun Xu
user
SherryXu
67d8eadc045f869fea1ce3f8
Yi Zhan
67d8eadc045f869fea1ce3f9
Börje F. Karlsson
claimed_verified
2025-03-18T08:07:34.005Z
61e52be53d6dbb1da842316a
https://cdn-avatars.huggingface.co/v1/production/uploads/61e52be53d6dbb1da842316a/gx0WGPcOCClXPymoKglc4.jpeg
Börje Karlsson
user
tellarin
67d8eadc045f869fea1ce3fa
Zongqing Lu
admin_assigned
2025-03-18T09:16:58.409Z
67d905c0e27ba28109384f5c
/avatars/26712594ac9d43c8d1a3e75e36b5df16.svg
Zongqing Lu
user
chungtsing
Being-0: A Humanoid Robotic Agent with Vision-Language Models and
  Modular Skills
Building autonomous robotic agents capable of achieving human-level
performance in real-world embodied tasks is an ultimate goal in humanoid robot
research. Recent advances have made significant progress in high-level
cognition with Foundation Models (FMs) and low-level skill development for
humanoid robots. However, directly combining these components often results in
poor robustness and efficiency due to compounding errors in long-horizon tasks
and the varied latency of different modules. We introduce Being-0, a
hierarchical agent framework that integrates an FM with a modular skill
library. The FM handles high-level cognitive tasks such as instruction
understanding, task planning, and reasoning, while the skill library provides
stable locomotion and dexterous manipulation for low-level control. To bridge
the gap between these levels, we propose a novel Connector module, powered by a
lightweight vision-language model (VLM). The Connector enhances the FM's
embodied capabilities by translating language-based plans into actionable skill
commands and dynamically coordinating locomotion and manipulation to improve
task success. With all components, except the FM, deployable on low-cost
onboard computation devices, Being-0 achieves efficient, real-time performance
on a full-sized humanoid robot equipped with dexterous hands and active vision.
Extensive experiments in large indoor environments demonstrate Being-0's
effectiveness in solving complex, long-horizon tasks that require challenging
navigation and manipulation subtasks. For further details and videos, visit
https://beingbeyond.github.io/being-0.
Being-0, a hierarchical framework integrating Foundation Models with a modular skill library and a lightweight vision-language model, achieves efficient real-time performance on complex humanoid robot tasks by coordinating high-level cognition with low-level control.
Foundation Models hierarchical agent framework modular skill library lightweight vision-language model vision-language model Connector module