Grasping Diverse Objects with Simulated Humanoids
We present a method for controlling a simulated humanoid to grasp an object
and move it to follow an object trajectory. Due to the challenges in
controlling a humanoid with dexterous hands, prior methods often use a
disembodied hand and only consider vertical lifts or short trajectories. This
limited scope hampers their applicability for object manipulation required for
animation and simulation. To close this gap, we learn a controller that can
pick up a large number (>1200) of objects and carry them to follow randomly
generated trajectories. Our key insight is to leverage a humanoid motion
representation that provides human-like motor skills and significantly speeds
up training. Using only simplistic reward, state, and object representations,
our method shows favorable scalability on diverse object and trajectories. For
training, we do not need dataset of paired full-body motion and object
trajectories. At test time, we only require the object mesh and desired
trajectories for grasping and transporting. To demonstrate the capabilities of
our method, we show state-of-the-art success rates in following object
trajectories and generalizing to unseen objects. Code and models will be
released.
669735ac84248e122c79ce65
Zhengyi Luo
claimed_verified
2024-07-23T07:48:27.014Z
645eaf37f0f92653b9f2ba77
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/VM-hXdknuRfeK3BD9FbyR.png
Luo
user
Zhengyiluo
669735ac84248e122c79ce66
Jinkun Cao
669735ac84248e122c79ce67
Sammy Christen
669735ac84248e122c79ce68
Alexander Winkler
669735ac84248e122c79ce69
Kris Kitani
669735ac84248e122c79ce6a
Weipeng Xu
Grasping Diverse Objects with Simulated Humanoids
We present a method for controlling a simulated humanoid to grasp an object
and move it to follow an object trajectory. Due to the challenges in
controlling a humanoid with dexterous hands, prior methods often use a
disembodied hand and only consider vertical lifts or short trajectories. This
limited scope hampers their applicability for object manipulation required for
animation and simulation. To close this gap, we learn a controller that can
pick up a large number (>1200) of objects and carry them to follow randomly
generated trajectories. Our key insight is to leverage a humanoid motion
representation that provides human-like motor skills and significantly speeds
up training. Using only simplistic reward, state, and object representations,
our method shows favorable scalability on diverse object and trajectories. For
training, we do not need dataset of paired full-body motion and object
trajectories. At test time, we only require the object mesh and desired
trajectories for grasping and transporting. To demonstrate the capabilities of
our method, we show state-of-the-art success rates in following object
trajectories and generalizing to unseen objects. Code and models will be
released.
A humanoid motion representation enables a versatile controller to grasp and transport objects following diverse trajectories with high success rates.
humanoid motion representation controller grasping object manipulation object trajectories state-of-the-art generalization