RT-H: Action Hierarchies Using Language
Language provides a way to break down complex concepts into digestible
pieces. Recent works in robot imitation learning use language-conditioned
policies that predict actions given visual observations and the high-level task
specified in language. These methods leverage the structure of natural language
to share data between semantically similar tasks (e.g., "pick coke can" and
"pick an apple") in multi-task datasets. However, as tasks become more
semantically diverse (e.g., "pick coke can" and "pour cup"), sharing data
between tasks becomes harder, so learning to map high-level tasks to actions
requires much more demonstration data. To bridge tasks and actions, our insight
is to teach the robot the language of actions, describing low-level motions
with more fine-grained phrases like "move arm forward". Predicting these
language motions as an intermediate step between tasks and actions forces the
policy to learn the shared structure of low-level motions across seemingly
disparate tasks. Furthermore, a policy that is conditioned on language motions
can easily be corrected during execution through human-specified language
motions. This enables a new paradigm for flexible policies that can learn from
human intervention in language. Our method RT-H builds an action hierarchy
using language motions: it first learns to predict language motions, and
conditioned on this and the high-level task, it predicts actions, using visual
context at all stages. We show that RT-H leverages this language-action
hierarchy to learn policies that are more robust and flexible by effectively
tapping into multi-task datasets. We show that these policies not only allow
for responding to language interventions, but can also learn from such
interventions and outperform methods that learn from teleoperated
interventions. Our website and videos are found at
https://rt-hierarchy.github.io.
65e7561fe35dc17c217bd940
Suneel Belkhale
claimed_verified
2024-07-31T07:45:28.008Z
66a922488f5cada4a6cb7567
/avatars/eab42b6ed39a0211c7ac13c88e39d7b2.svg
Suneel Belkhale
user
belkhale
65e7561fe35dc17c217bd941
Tianli Ding
65e7561fe35dc17c217bd942
Ted Xiao
65e7561fe35dc17c217bd943
Pierre Sermanet
65e7561fe35dc17c217bd944
Quon Vuong
65e7561fe35dc17c217bd945
Jonathan Tompson
65e7561fe35dc17c217bd946
Yevgen Chebotar
admin_assigned
2024-03-05T21:03:13.353Z
64c93f9c9471c54a9438162c
https://cdn-avatars.huggingface.co/v1/production/uploads/64c93f9c9471c54a9438162c/bYGUqFsF67VuAuXecjxjX.jpeg
Yevgen Chebotar
user
ychebotar
65e7561fe35dc17c217bd947
Debidatta Dwibedi
65e7561fe35dc17c217bd948
Dorsa Sadigh
RT-H: Action Hierarchies Using Language
Language provides a way to break down complex concepts into digestible
pieces. Recent works in robot imitation learning use language-conditioned
policies that predict actions given visual observations and the high-level task
specified in language. These methods leverage the structure of natural language
to share data between semantically similar tasks (e.g., "pick coke can" and
"pick an apple") in multi-task datasets. However, as tasks become more
semantically diverse (e.g., "pick coke can" and "pour cup"), sharing data
between tasks becomes harder, so learning to map high-level tasks to actions
requires much more demonstration data. To bridge tasks and actions, our insight
is to teach the robot the language of actions, describing low-level motions
with more fine-grained phrases like "move arm forward". Predicting these
language motions as an intermediate step between tasks and actions forces the
policy to learn the shared structure of low-level motions across seemingly
disparate tasks. Furthermore, a policy that is conditioned on language motions
can easily be corrected during execution through human-specified language
motions. This enables a new paradigm for flexible policies that can learn from
human intervention in language. Our method RT-H builds an action hierarchy
using language motions: it first learns to predict language motions, and
conditioned on this and the high-level task, it predicts actions, using visual
context at all stages. We show that RT-H leverages this language-action
hierarchy to learn policies that are more robust and flexible by effectively
tapping into multi-task datasets. We show that these policies not only allow
for responding to language interventions, but can also learn from such
interventions and outperform methods that learn from teleoperated
interventions. Our website and videos are found at
https://rt-hierarchy.github.io.
A method RT-H uses language-motion hierarchies to improve robot imitation learning by predicting low-level actions from high-level language tasks, enhancing flexibility and robustness.
language-conditioned policies action prediction visual observations multi-task datasets semantic diversity language of actions low-level motions language motions action hierarchy human-specified language motions language interventions teleoperated interventions