DexUMI: Using Human Hand as the Universal Manipulation Interface for
  Dexterous Manipulation
We present DexUMI - a data collection and policy learning framework that uses
the human hand as the natural interface to transfer dexterous manipulation
skills to various robot hands. DexUMI includes hardware and software
adaptations to minimize the embodiment gap between the human hand and various
robot hands. The hardware adaptation bridges the kinematics gap using a
wearable hand exoskeleton. It allows direct haptic feedback in manipulation
data collection and adapts human motion to feasible robot hand motion. The
software adaptation bridges the visual gap by replacing the human hand in video
data with high-fidelity robot hand inpainting. We demonstrate DexUMI's
capabilities through comprehensive real-world experiments on two different
dexterous robot hand hardware platforms, achieving an average task success rate
of 86%.
683b8af5091615f46fabadde
Mengda Xu
claimed_verified
2025-06-02T07:42:10.223Z
655a50a850b9a14799165d53
/avatars/6bb37ac0b6840771ff7a6a6da4192ace.svg
Mengda Xu
user
mengdaxu
683b8af5091615f46fabaddf
Han Zhang
683b8af5091615f46fabade0
Yifan Hou
683b8af5091615f46fabade1
Zhenjia Xu
683b8af5091615f46fabade2
Linxi Fan
683b8af5091615f46fabade3
Manuela Veloso
683b8af5091615f46fabade4
Shuran Song
DexUMI: Using Human Hand as the Universal Manipulation Interface for
  Dexterous Manipulation
We present DexUMI - a data collection and policy learning framework that uses
the human hand as the natural interface to transfer dexterous manipulation
skills to various robot hands. DexUMI includes hardware and software
adaptations to minimize the embodiment gap between the human hand and various
robot hands. The hardware adaptation bridges the kinematics gap using a
wearable hand exoskeleton. It allows direct haptic feedback in manipulation
data collection and adapts human motion to feasible robot hand motion. The
software adaptation bridges the visual gap by replacing the human hand in video
data with high-fidelity robot hand inpainting. We demonstrate DexUMI's
capabilities through comprehensive real-world experiments on two different
dexterous robot hand hardware platforms, achieving an average task success rate
of 86%.
DexUMI framework utilizes a wearable hand exoskeleton and high-fidelity robot hand inpainting to transfer dexterous manipulation skills from human hands to robot hands, achieving high task success rates.
wearable hand exoskeleton haptic feedback robot hand inpainting dexterous manipulation kinematics visual gap