RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robots
Recent advancements in Artificial Intelligence (AI) have largely been
propelled by scaling. In Robotics, scaling is hindered by the lack of access to
massive robot datasets. We advocate using realistic physical simulation as a
means to scale environments, tasks, and datasets for robot learning methods. We
present RoboCasa, a large-scale simulation framework for training generalist
robots in everyday environments. RoboCasa features realistic and diverse scenes
focusing on kitchen environments. We provide thousands of 3D assets across over
150 object categories and dozens of interactable furniture and appliances. We
enrich the realism and diversity of our simulation with generative AI tools,
such as object assets from text-to-3D models and environment textures from
text-to-image models. We design a set of 100 tasks for systematic evaluation,
including composite tasks generated by the guidance of large language models.
To facilitate learning, we provide high-quality human demonstrations and
integrate automated trajectory generation methods to substantially enlarge our
datasets with minimal human burden. Our experiments show a clear scaling trend
in using synthetically generated robot data for large-scale imitation learning
and show great promise in harnessing simulation data in real-world tasks.
Videos and open-source code are available at https://robocasa.ai/
665fe6f5b4fb1c0cb09d12ed
Soroush Nasiriany
665fe6f5b4fb1c0cb09d12ee
Abhiram Maddukuri
admin_assigned
2024-06-05T09:39:36.510Z
63a753248729ce5b5642e096
https://cdn-avatars.huggingface.co/v1/production/uploads/1671910128497-noauth.jpeg
Abhiram Maddukuri
user
Abhi03
665fe6f5b4fb1c0cb09d12ef
Lance Zhang
admin_assigned
2024-06-05T09:39:43.772Z
64cce61fc176cd9dfa950b02
/avatars/8b67572bf9ef271d7f5f5e46ff2b6fee.svg
Lance Zhang
user
LanceZhang
665fe6f5b4fb1c0cb09d12f0
Adeet Parikh
665fe6f5b4fb1c0cb09d12f1
Aaron Lo
665fe6f5b4fb1c0cb09d12f2
Abhishek Joshi
665fe6f5b4fb1c0cb09d12f3
Ajay Mandlekar
admin_assigned
2024-06-05T09:40:25.049Z
653ab89a7c01c693a17d88f7
/avatars/30f3428d5cd04b2926b80c806126d827.svg
Ajay Mandlekar
user
amandlek
665fe6f5b4fb1c0cb09d12f4
Yuke Zhu
RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robots
Recent advancements in Artificial Intelligence (AI) have largely been
propelled by scaling. In Robotics, scaling is hindered by the lack of access to
massive robot datasets. We advocate using realistic physical simulation as a
means to scale environments, tasks, and datasets for robot learning methods. We
present RoboCasa, a large-scale simulation framework for training generalist
robots in everyday environments. RoboCasa features realistic and diverse scenes
focusing on kitchen environments. We provide thousands of 3D assets across over
150 object categories and dozens of interactable furniture and appliances. We
enrich the realism and diversity of our simulation with generative AI tools,
such as object assets from text-to-3D models and environment textures from
text-to-image models. We design a set of 100 tasks for systematic evaluation,
including composite tasks generated by the guidance of large language models.
To facilitate learning, we provide high-quality human demonstrations and
integrate automated trajectory generation methods to substantially enlarge our
datasets with minimal human burden. Our experiments show a clear scaling trend
in using synthetically generated robot data for large-scale imitation learning
and show great promise in harnessing simulation data in real-world tasks.
Videos and open-source code are available at https://robocasa.ai/
RoboCasa is a large-scale simulation framework using realistic physical environments, diverse tasks, and datasets enhanced with generative AI for training generalist robots.
physical simulation large-scale simulation framework kitchen environments 3D assets text-to-3D models text-to-image models systematic evaluation human demonstrations automated trajectory generation large-scale imitation learning