RP1M: A Large-Scale Motion Dataset for Piano Playing with Bi-Manual
  Dexterous Robot Hands
It has been a long-standing research goal to endow robot hands with
human-level dexterity. Bi-manual robot piano playing constitutes a task that
combines challenges from dynamic tasks, such as generating fast while precise
motions, with slower but contact-rich manipulation problems. Although
reinforcement learning based approaches have shown promising results in
single-task performance, these methods struggle in a multi-song setting. Our
work aims to close this gap and, thereby, enable imitation learning approaches
for robot piano playing at scale. To this end, we introduce the Robot Piano 1
Million (RP1M) dataset, containing bi-manual robot piano playing motion data of
more than one million trajectories. We formulate finger placements as an
optimal transport problem, thus, enabling automatic annotation of vast amounts
of unlabeled songs. Benchmarking existing imitation learning approaches shows
that such approaches reach state-of-the-art robot piano playing performance by
leveraging RP1M.
66c5593fb666ae525586a3ea
Yi Zhao
66c5593fb666ae525586a3eb
Le Chen
66c5593fb666ae525586a3ec
Jan Schneider
66c5593fb666ae525586a3ed
Quankai Gao
66c5593fb666ae525586a3ee
Juho Kannala
66c5593fb666ae525586a3ef
Bernhard Schölkopf
66c5593fb666ae525586a3f0
Joni Pajarinen
66c5593fb666ae525586a3f1
Dieter Büchler
RP1M: A Large-Scale Motion Dataset for Piano Playing with Bi-Manual
  Dexterous Robot Hands
It has been a long-standing research goal to endow robot hands with
human-level dexterity. Bi-manual robot piano playing constitutes a task that
combines challenges from dynamic tasks, such as generating fast while precise
motions, with slower but contact-rich manipulation problems. Although
reinforcement learning based approaches have shown promising results in
single-task performance, these methods struggle in a multi-song setting. Our
work aims to close this gap and, thereby, enable imitation learning approaches
for robot piano playing at scale. To this end, we introduce the Robot Piano 1
Million (RP1M) dataset, containing bi-manual robot piano playing motion data of
more than one million trajectories. We formulate finger placements as an
optimal transport problem, thus, enabling automatic annotation of vast amounts
of unlabeled songs. Benchmarking existing imitation learning approaches shows
that such approaches reach state-of-the-art robot piano playing performance by
leveraging RP1M.
A dataset of over one million bi-manual robot piano playing trajectories enables state-of-the-art imitation learning performance for robot piano playing.
bi-manual robot piano playing reinforcement learning imitation learning RP1M dataset finger placements optimal transport problem