Guided Optimal Transport for Sim-and-Real Policy Co-Training
Shuo Cheng, Liqian Ma, Zhenyang Chen, Ajay Mandlekar, Caelan Garrett, Danfei Xu
Behavior cloning has shown promise for robot manipulation by mimicking human demonstrations, but achieving robust, generalizable performance in the real world often requires costly and labor-intensive data collection to obtain these demonstrations. Recent advances in simulation and automated motion synthesis offer scalable alternatives for generating training data. However, transferring policies from simulation to the real world remains challenging due to simulation modeling inaccuracies. In this work, we propose a framework for learning generalizable manipulation policies that primarily leverages simulation and only requires a few real-world demonstrations. Central to our approach is learning a shared feature space that preserves task-relevant structure across simulation and the real world. Specifically, augment traditional imitation learning objective functions with a new loss inspired by optimal transport that encourages domain-invariant feature learning. We pair this with a motion generator that automatically synthesizes diverse simulated trajectories from a few manual demonstrations. We validate our method on challenging manipulation tasks in both simulation, where investigate sim-to-sim transfer, and the real world, demonstrating effective and data-efficient policy transfer.