AnyTeleop: A General Vision-Based Dexterous Robot Arm-Hand Teleoperation
  System
Vision-based teleoperation offers the possibility to endow robots with
human-level intelligence to physically interact with the environment, while
only requiring low-cost camera sensors. However, current vision-based
teleoperation systems are designed and engineered towards a particular robot
model and deploy environment, which scales poorly as the pool of the robot
models expands and the variety of the operating environment increases. In this
paper, we propose AnyTeleop, a unified and general teleoperation system to
support multiple different arms, hands, realities, and camera configurations
within a single system. Although being designed to provide great flexibility to
the choice of simulators and real hardware, our system can still achieve great
performance. For real-world experiments, AnyTeleop can outperform a previous
system that was designed for a specific robot hardware with a higher success
rate, using the same robot. For teleoperation in simulation, AnyTeleop leads to
better imitation learning performance, compared with a previous system that is
particularly designed for that simulator. Project page: http://anyteleop.com/.
64ace52bc4e03b989f7fe5d1
Yuzhe Qin
claimed_verified
2023-07-12T07:29:06.672Z
64ad64c2c1da7c4dbc7b19c5
/avatars/98efbab6c65405e8701c64c290a7a87c.svg
Yuzhe Qin
user
yuzheqin
64ace52bc4e03b989f7fe5d2
Wei Yang
64ace52bc4e03b989f7fe5d3
Binghao Huang
claimed_verified
2023-07-13T08:32:55.325Z
64af4da473151ec37ee461a5
/avatars/fea36cd0fa995c9827edce40286b9245.svg
Binghao Huang
user
Binghaohuang
64ace52bc4e03b989f7fe5d4
Karl Van Wyk
64ace52bc4e03b989f7fe5d5
Hao Su
64ace52bc4e03b989f7fe5d6
Xiaolong Wang
64ace52bc4e03b989f7fe5d7
Yu-Wei Chao
64ace52bc4e03b989f7fe5d8
Dietor Fox
AnyTeleop: A General Vision-Based Dexterous Robot Arm-Hand Teleoperation
  System
Vision-based teleoperation offers the possibility to endow robots with
human-level intelligence to physically interact with the environment, while
only requiring low-cost camera sensors. However, current vision-based
teleoperation systems are designed and engineered towards a particular robot
model and deploy environment, which scales poorly as the pool of the robot
models expands and the variety of the operating environment increases. In this
paper, we propose AnyTeleop, a unified and general teleoperation system to
support multiple different arms, hands, realities, and camera configurations
within a single system. Although being designed to provide great flexibility to
the choice of simulators and real hardware, our system can still achieve great
performance. For real-world experiments, AnyTeleop can outperform a previous
system that was designed for a specific robot hardware with a higher success
rate, using the same robot. For teleoperation in simulation, AnyTeleop leads to
better imitation learning performance, compared with a previous system that is
particularly designed for that simulator. Project page: http://anyteleop.com/.
AnyTeleop is a flexible and unified teleoperation system that supports various robotic and environmental configurations, achieving superior performance compared to specialized systems in both real-world and simulated environments.
vision-based teleoperation human-level intelligence low-cost camera sensors unified teleoperation system multiple robot models flexible simulators real hardware real-world experiments imitation learning simulation