IRASim: Learning Interactive Real-Robot Action Simulators
Scalable robot learning in the real world is limited by the cost and safety
issues of real robots. In addition, rolling out robot trajectories in the real
world can be time-consuming and labor-intensive. In this paper, we propose to
learn an interactive real-robot action simulator as an alternative. We
introduce a novel method, IRASim, which leverages the power of generative
models to generate extremely realistic videos of a robot arm that executes a
given action trajectory, starting from an initial given frame. To validate the
effectiveness of our method, we create a new benchmark, IRASim Benchmark, based
on three real-robot datasets and perform extensive experiments on the
benchmark. Results show that IRASim outperforms all the baseline methods and is
more preferable in human evaluations. We hope that IRASim can serve as an
effective and scalable approach to enhance robot learning in the real world. To
promote research for generative real-robot action simulators, we open-source
code, benchmark, and checkpoints at https: //gen-irasim.github.io.
6679a8667dc34fe17cb367f1
Fangqi Zhu
extracted_confirmed
2024-06-25T06:22:06.567Z
64d0b110d1fda042b57a3c0b
/avatars/0cf0050b272d1c8e46e1299de2aea22a.svg
zhu
user
fangqi
6679a8667dc34fe17cb367f2
Hongtao Wu
admin_assigned
2024-06-25T13:52:17.258Z
667094befc615a62578c7a87
/avatars/4e645f8cff38b0b6e0d1016cd5a23d89.svg
Hongtao Wu
user
danchaoren
6679a8667dc34fe17cb367f3
Song Guo
6679a8667dc34fe17cb367f4
Yuxiao Liu
6679a8667dc34fe17cb367f5
Chilam Cheang
6679a8667dc34fe17cb367f6
Tao Kong
admin_assigned
2024-06-25T13:52:48.429Z
6618de1782c27355a197b47f
/avatars/ba746df3ae4d264af6184523930fba1c.svg
Kong
user
TaoKong
IRASim: Learning Interactive Real-Robot Action Simulators
Scalable robot learning in the real world is limited by the cost and safety
issues of real robots. In addition, rolling out robot trajectories in the real
world can be time-consuming and labor-intensive. In this paper, we propose to
learn an interactive real-robot action simulator as an alternative. We
introduce a novel method, IRASim, which leverages the power of generative
models to generate extremely realistic videos of a robot arm that executes a
given action trajectory, starting from an initial given frame. To validate the
effectiveness of our method, we create a new benchmark, IRASim Benchmark, based
on three real-robot datasets and perform extensive experiments on the
benchmark. Results show that IRASim outperforms all the baseline methods and is
more preferable in human evaluations. We hope that IRASim can serve as an
effective and scalable approach to enhance robot learning in the real world. To
promote research for generative real-robot action simulators, we open-source
code, benchmark, and checkpoints at https: //gen-irasim.github.io.
IRASim utilizes generative models to create realistic robot action simulations, enhancing scalable robot learning by reducing the need for real-world testing and increasing efficiency.
generative models robot action simulator real-world robot learning benchmark human evaluations