OmniRetarget: Interaction-Preserving Data Generation for Humanoid
  Whole-Body Loco-Manipulation and Scene Interaction
A dominant paradigm for teaching humanoid robots complex skills is to
retarget human motions as kinematic references to train reinforcement learning
(RL) policies. However, existing retargeting pipelines often struggle with the
significant embodiment gap between humans and robots, producing physically
implausible artifacts like foot-skating and penetration. More importantly,
common retargeting methods neglect the rich human-object and human-environment
interactions essential for expressive locomotion and loco-manipulation. To
address this, we introduce OmniRetarget, an interaction-preserving data
generation engine based on an interaction mesh that explicitly models and
preserves the crucial spatial and contact relationships between an agent, the
terrain, and manipulated objects. By minimizing the Laplacian deformation
between the human and robot meshes while enforcing kinematic constraints,
OmniRetarget generates kinematically feasible trajectories. Moreover,
preserving task-relevant interactions enables efficient data augmentation, from
a single demonstration to different robot embodiments, terrains, and object
configurations. We comprehensively evaluate OmniRetarget by retargeting motions
from OMOMO, LAFAN1, and our in-house MoCap datasets, generating over 8-hour
trajectories that achieve better kinematic constraint satisfaction and contact
preservation than widely used baselines. Such high-quality data enables
proprioceptive RL policies to successfully execute long-horizon (up to 30
seconds) parkour and loco-manipulation skills on a Unitree G1 humanoid, trained
with only 5 reward terms and simple domain randomization shared by all tasks,
without any learning curriculum.
68e94146b12ca14cb8a165e7
Lujie Yang
68e94146b12ca14cb8a165e8
Xiaoyu Huang
68e94146b12ca14cb8a165e9
Zhen Wu
68e94146b12ca14cb8a165ea
Angjoo Kanazawa
68e94146b12ca14cb8a165eb
Pieter Abbeel
68e94146b12ca14cb8a165ec
Carmelo Sferrazza
68e94146b12ca14cb8a165ed
C. Karen Liu
68e94146b12ca14cb8a165ee
Rocky Duan
68e94146b12ca14cb8a165ef
Guanya Shi
OmniRetarget: Interaction-Preserving Data Generation for Humanoid
  Whole-Body Loco-Manipulation and Scene Interaction
A dominant paradigm for teaching humanoid robots complex skills is to
retarget human motions as kinematic references to train reinforcement learning
(RL) policies. However, existing retargeting pipelines often struggle with the
significant embodiment gap between humans and robots, producing physically
implausible artifacts like foot-skating and penetration. More importantly,
common retargeting methods neglect the rich human-object and human-environment
interactions essential for expressive locomotion and loco-manipulation. To
address this, we introduce OmniRetarget, an interaction-preserving data
generation engine based on an interaction mesh that explicitly models and
preserves the crucial spatial and contact relationships between an agent, the
terrain, and manipulated objects. By minimizing the Laplacian deformation
between the human and robot meshes while enforcing kinematic constraints,
OmniRetarget generates kinematically feasible trajectories. Moreover,
preserving task-relevant interactions enables efficient data augmentation, from
a single demonstration to different robot embodiments, terrains, and object
configurations. We comprehensively evaluate OmniRetarget by retargeting motions
from OMOMO, LAFAN1, and our in-house MoCap datasets, generating over 8-hour
trajectories that achieve better kinematic constraint satisfaction and contact
preservation than widely used baselines. Such high-quality data enables
proprioceptive RL policies to successfully execute long-horizon (up to 30
seconds) parkour and loco-manipulation skills on a Unitree G1 humanoid, trained
with only 5 reward terms and simple domain randomization shared by all tasks,
without any learning curriculum.
OmniRetarget generates high-quality, interaction-preserving motion data for training RL policies, enabling complex skills like parkour and loco-manipulation on humanoid robots.
reinforcement learning (RL) retargeting embodiment gap foot-skating penetration human-object interactions human-environment interactions interaction mesh Laplacian deformation kinematic constraints data augmentation proprioceptive RL policies long-horizon skills parkour loco-manipulation Unitree G1 reward terms domain randomization