Being-H0: Vision-Language-Action Pretraining from Large-Scale Human
  Videos
We introduce Being-H0, a dexterous Vision-Language-Action model (VLA) trained
on large-scale human videos. Existing VLAs struggle with complex manipulation
tasks requiring high dexterity and generalize poorly to novel scenarios and
tasks, primarily due to their reliance on synthetic data with significant
sim-to-real gaps or teleoperated demonstrations lacking scale and diversity. To
address this data bottleneck, we propose leveraging human hands as a foundation
manipulator, capitalizing on the rich dexterity and scalability present in web
data. Our approach centers on physical instruction tuning, a novel training
paradigm that combines large-scale VLA pretraining from human videos, physical
space alignment for 3D reasoning, and post-training adaptation for robotic
tasks. Additionally, we introduce a part-level motion tokenization method which
achieves millimeter-level reconstruction accuracy to model precise hand
trajectories for action learning. To support our proposed paradigm, we further
develop a comprehensive data curation pipeline that integrates heterogeneous
sources -- including motion capture, VR, and RGB-only videos -- into a
large-scale dataset with millions of motion-based instructional instances. We
empirically show the excellence of Being-H0 in hand motion generation and
instruction following, and it also scales well with model and data sizes.
Importantly, we observe the expected gains of Being-H0 in real-world robotic
manipulation as physical instruction tuning is applied. More details are
available at https://beingbeyond.github.io/Being-H0.
687f0cd433947f780d9b4b29
Hao Luo
687f0cd433947f780d9b4b2a
Yicheng Feng
687f0cd433947f780d9b4b2b
Wanpeng Zhang
claimed_verified
2025-07-22T07:47:51.137Z
640dd700fdeaae139081f598
https://cdn-avatars.huggingface.co/v1/production/uploads/640dd700fdeaae139081f598/L986zu4-iOPFs9Y3_T5Ue.jpeg
Wanpeng Zhang
user
zawnpn
687f0cd433947f780d9b4b2c
Sipeng Zheng
687f0cd433947f780d9b4b2d
Ye Wang
687f0cd433947f780d9b4b2e
Haoqi Yuan
claimed_verified
2025-07-22T08:33:30.666Z
644560657a7b94ddc2d445a3
/avatars/09d6447da6ff1bd0b2b00c899c9f1b28.svg
Haoqi Yuan
user
Yaya041
687f0cd433947f780d9b4b2f
Jiazheng Liu
687f0cd433947f780d9b4b30
Chaoyi Xu
687f0cd433947f780d9b4b31
Qin Jin
687f0cd433947f780d9b4b32
Zongqing Lu
Being-H0: Vision-Language-Action Pretraining from Large-Scale Human
  Videos
We introduce Being-H0, a dexterous Vision-Language-Action model (VLA) trained
on large-scale human videos. Existing VLAs struggle with complex manipulation
tasks requiring high dexterity and generalize poorly to novel scenarios and
tasks, primarily due to their reliance on synthetic data with significant
sim-to-real gaps or teleoperated demonstrations lacking scale and diversity. To
address this data bottleneck, we propose leveraging human hands as a foundation
manipulator, capitalizing on the rich dexterity and scalability present in web
data. Our approach centers on physical instruction tuning, a novel training
paradigm that combines large-scale VLA pretraining from human videos, physical
space alignment for 3D reasoning, and post-training adaptation for robotic
tasks. Additionally, we introduce a part-level motion tokenization method which
achieves millimeter-level reconstruction accuracy to model precise hand
trajectories for action learning. To support our proposed paradigm, we further
develop a comprehensive data curation pipeline that integrates heterogeneous
sources -- including motion capture, VR, and RGB-only videos -- into a
large-scale dataset with millions of motion-based instructional instances. We
empirically show the excellence of Being-H0 in hand motion generation and
instruction following, and it also scales well with model and data sizes.
Importantly, we observe the expected gains of Being-H0 in real-world robotic
manipulation as physical instruction tuning is applied. More details are
available at https://beingbeyond.github.io/Being-H0.
Being-H0, a Vision-Language-Action model, uses human videos and physical instruction tuning to achieve high dexterity in manipulation tasks and scales well with data and model size.
Vision-Language-Action model dexterous manipulation synthetic data teleoperated demonstrations physical instruction tuning 3D reasoning part-level motion tokenization motion capture VR RGB-only videos hand motion generation instruction following real-world robotic manipulation