DexArt: Benchmarking Generalizable Dexterous Manipulation with
  Articulated Objects
To enable general-purpose robots, we will require the robot to operate daily
articulated objects as humans do. Current robot manipulation has heavily relied
on using a parallel gripper, which restricts the robot to a limited set of
objects. On the other hand, operating with a multi-finger robot hand will allow
better approximation to human behavior and enable the robot to operate on
diverse articulated objects. To this end, we propose a new benchmark called
DexArt, which involves Dexterous manipulation with Articulated objects in a
physical simulator. In our benchmark, we define multiple complex manipulation
tasks, and the robot hand will need to manipulate diverse articulated objects
within each task. Our main focus is to evaluate the generalizability of the
learned policy on unseen articulated objects. This is very challenging given
the high degrees of freedom of both hands and objects. We use Reinforcement
Learning with 3D representation learning to achieve generalization. Through
extensive studies, we provide new insights into how 3D representation learning
affects decision making in RL with 3D point cloud inputs. More details can be
found at https://www.chenbao.tech/dexart/.
645c61732e8e7c269fd944df
Chen Bao
claimed_verified
2025-01-15T08:52:10.086Z
676050fb1d80fb3c2083b9b7
/avatars/f5cdcb9fb026d53b796a4c9b5864927d.svg
Chen Bao
user
Kami-code
645c61732e8e7c269fd944e0
Helin Xu
admin_assigned
2023-06-02T14:45:00.013Z
647988332b688f67eec58f62
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/P24GTakbHzWOHHj0KksoZ.jpeg
Helin Xu
user
HelinXu
645c61732e8e7c269fd944e1
Yuzhe Qin
admin_assigned
2023-06-02T14:45:09.328Z
64765e133d8d1cb7918f494d
/avatars/17dd8e5920a21c90bdb55f193cc01d88.svg
Yuzhe Qin
user
yzqin
645c61732e8e7c269fd944e2
Xiaolong Wang
DexArt: Benchmarking Generalizable Dexterous Manipulation with
  Articulated Objects
To enable general-purpose robots, we will require the robot to operate daily
articulated objects as humans do. Current robot manipulation has heavily relied
on using a parallel gripper, which restricts the robot to a limited set of
objects. On the other hand, operating with a multi-finger robot hand will allow
better approximation to human behavior and enable the robot to operate on
diverse articulated objects. To this end, we propose a new benchmark called
DexArt, which involves Dexterous manipulation with Articulated objects in a
physical simulator. In our benchmark, we define multiple complex manipulation
tasks, and the robot hand will need to manipulate diverse articulated objects
within each task. Our main focus is to evaluate the generalizability of the
learned policy on unseen articulated objects. This is very challenging given
the high degrees of freedom of both hands and objects. We use Reinforcement
Learning with 3D representation learning to achieve generalization. Through
extensive studies, we provide new insights into how 3D representation learning
affects decision making in RL with 3D point cloud inputs. More details can be
found at https://www.chenbao.tech/dexart/.
A new benchmark, DexArt, is proposed to evaluate the generalizability of reinforcement learning policies in dexterous manipulation of articulated objects using a multi-finger robot hand in a physical simulator.
Reinforcement Learning 3D representation learning 3D point cloud inputs multi-finger robot hand articulated objects physical simulator