Universal Manipulation Interface: In-The-Wild Robot Teaching Without
  In-The-Wild Robots
We present Universal Manipulation Interface (UMI) -- a data collection and
policy learning framework that allows direct skill transfer from in-the-wild
human demonstrations to deployable robot policies. UMI employs hand-held
grippers coupled with careful interface design to enable portable, low-cost,
and information-rich data collection for challenging bimanual and dynamic
manipulation demonstrations. To facilitate deployable policy learning, UMI
incorporates a carefully designed policy interface with inference-time latency
matching and a relative-trajectory action representation. The resulting learned
policies are hardware-agnostic and deployable across multiple robot platforms.
Equipped with these features, UMI framework unlocks new robot manipulation
capabilities, allowing zero-shot generalizable dynamic, bimanual, precise, and
long-horizon behaviors, by only changing the training data for each task. We
demonstrate UMI's versatility and efficacy with comprehensive real-world
experiments, where policies learned via UMI zero-shot generalize to novel
environments and objects when trained on diverse human demonstrations. UMI's
hardware and software system is open-sourced at https://umi-gripper.github.io.
65d2edfab3eb7eed71f9e812
Cheng Chi
claimed_verified
2024-02-20T08:39:01.601Z
630440995c70c21d0eaea552
https://cdn-avatars.huggingface.co/v1/production/uploads/630440995c70c21d0eaea552/FxrpcUgnnnGWV_9k-KcBa.jpeg
Cheng Chi
user
chengchi
65d2edfab3eb7eed71f9e813
Zhenjia Xu
65d2edfab3eb7eed71f9e814
Chuer Pan
admin_assigned
2024-02-19T15:17:31.863Z
63efc1b3fc92a63ac8116ff2
/avatars/60af94291eba0a6b5b43c5c1b58fbe70.svg
Pan
user
Chuer
65d2edfab3eb7eed71f9e815
Eric Cousineau
admin_assigned
2024-02-19T15:17:56.614Z
630a4b191d1650ec2f8e81a1
/avatars/0335fe1020963add3d25f831c5b6b047.svg
Eric Cousineau
user
EricCousineau-TRI
65d2edfab3eb7eed71f9e816
Benjamin Burchfiel
65d2edfab3eb7eed71f9e817
Siyuan Feng
65d2edfab3eb7eed71f9e818
Russ Tedrake
65d2edfab3eb7eed71f9e819
Shuran Song
Universal Manipulation Interface: In-The-Wild Robot Teaching Without
  In-The-Wild Robots
We present Universal Manipulation Interface (UMI) -- a data collection and
policy learning framework that allows direct skill transfer from in-the-wild
human demonstrations to deployable robot policies. UMI employs hand-held
grippers coupled with careful interface design to enable portable, low-cost,
and information-rich data collection for challenging bimanual and dynamic
manipulation demonstrations. To facilitate deployable policy learning, UMI
incorporates a carefully designed policy interface with inference-time latency
matching and a relative-trajectory action representation. The resulting learned
policies are hardware-agnostic and deployable across multiple robot platforms.
Equipped with these features, UMI framework unlocks new robot manipulation
capabilities, allowing zero-shot generalizable dynamic, bimanual, precise, and
long-horizon behaviors, by only changing the training data for each task. We
demonstrate UMI's versatility and efficacy with comprehensive real-world
experiments, where policies learned via UMI zero-shot generalize to novel
environments and objects when trained on diverse human demonstrations. UMI's
hardware and software system is open-sourced at https://umi-gripper.github.io.
UMI is a framework for transferring human manipulation skills to robots using hardware-agnostic policies that generalize well to new tasks and environments.
Universal Manipulation Interface data collection policy learning hand-held grippers bimanual manipulation dynamic manipulation policy interface inference-time latency matching relative-trajectory action representation zero-shot generalization