TidyBot: Personalized Robot Assistance with Large Language Models
For a robot to personalize physical assistance effectively, it must learn
user preferences that can be generally reapplied to future scenarios. In this
work, we investigate personalization of household cleanup with robots that can
tidy up rooms by picking up objects and putting them away. A key challenge is
determining the proper place to put each object, as people's preferences can
vary greatly depending on personal taste or cultural background. For instance,
one person may prefer storing shirts in the drawer, while another may prefer
them on the shelf. We aim to build systems that can learn such preferences from
just a handful of examples via prior interactions with a particular person. We
show that robots can combine language-based planning and perception with the
few-shot summarization capabilities of large language models (LLMs) to infer
generalized user preferences that are broadly applicable to future
interactions. This approach enables fast adaptation and achieves 91.2% accuracy
on unseen objects in our benchmark dataset. We also demonstrate our approach on
a real-world mobile manipulator called TidyBot, which successfully puts away
85.0% of objects in real-world test scenarios.
645afad0333fb183578705a0
Jimmy Wu
claimed_verified
2023-05-11T07:03:03.205Z
62faae5686456e533453422c
/avatars/78e175e131fe472257c2f93787e463a1.svg
Jimmy Wu
user
jimmyyhwu
645afad0333fb183578705a1
Rika Antonova
645afad0333fb183578705a2
Adam Kan
claimed_verified
2025-07-07T11:28:19.103Z
6827d31a452ad242a61558d2
/avatars/b5b8c72caf5d44ab8b396b20da5c98fa.svg
Adam Kan
user
adakan4
645afad0333fb183578705a3
Marion Lepert
extracted_confirmed
2023-05-10T02:01:35.028Z
63d9b55faefaa735258522f5
/avatars/5b5eb3dc922484db2d2646f0a435b338.svg
Marion Lepert
user
lepertm
645afad0333fb183578705a4
Andy Zeng
645afad0333fb183578705a5
Shuran Song
645afad0333fb183578705a6
Jeannette Bohg
claimed_verified
2024-12-18T09:55:47.853Z
676272a59865895e253ceee5
https://cdn-avatars.huggingface.co/v1/production/uploads/676272a59865895e253ceee5/Nz51Y-G49SD19lQEjH6Yd.jpeg
Jeannette Bohg
user
jbohg
645afad0333fb183578705a7
Szymon Rusinkiewicz
645afad0333fb183578705a8
Thomas Funkhouser
TidyBot: Personalized Robot Assistance with Large Language Models
For a robot to personalize physical assistance effectively, it must learn
user preferences that can be generally reapplied to future scenarios. In this
work, we investigate personalization of household cleanup with robots that can
tidy up rooms by picking up objects and putting them away. A key challenge is
determining the proper place to put each object, as people's preferences can
vary greatly depending on personal taste or cultural background. For instance,
one person may prefer storing shirts in the drawer, while another may prefer
them on the shelf. We aim to build systems that can learn such preferences from
just a handful of examples via prior interactions with a particular person. We
show that robots can combine language-based planning and perception with the
few-shot summarization capabilities of large language models (LLMs) to infer
generalized user preferences that are broadly applicable to future
interactions. This approach enables fast adaptation and achieves 91.2% accuracy
on unseen objects in our benchmark dataset. We also demonstrate our approach on
a real-world mobile manipulator called TidyBot, which successfully puts away
85.0% of objects in real-world test scenarios.
Robots can learn user preferences for household cleanup using few-shot language-based planning and perception with large language models, achieving high accuracy in both simulated and real-world scenarios.
language-based planning perception few-shot summarization large language models LLMs