GOAT: GO to Any Thing
In deployment scenarios such as homes and warehouses, mobile robots are
expected to autonomously navigate for extended periods, seamlessly executing
tasks articulated in terms that are intuitively understandable by human
operators. We present GO To Any Thing (GOAT), a universal navigation system
capable of tackling these requirements with three key features: a) Multimodal:
it can tackle goals specified via category labels, target images, and language
descriptions, b) Lifelong: it benefits from its past experience in the same
environment, and c) Platform Agnostic: it can be quickly deployed on robots
with different embodiments. GOAT is made possible through a modular system
design and a continually augmented instance-aware semantic memory that keeps
track of the appearance of objects from different viewpoints in addition to
category-level semantics. This enables GOAT to distinguish between different
instances of the same category to enable navigation to targets specified by
images and language descriptions. In experimental comparisons spanning over 90
hours in 9 different homes consisting of 675 goals selected across 200+
different object instances, we find GOAT achieves an overall success rate of
83%, surpassing previous methods and ablations by 32% (absolute improvement).
GOAT improves with experience in the environment, from a 60% success rate at
the first goal to a 90% success after exploration. In addition, we demonstrate
that GOAT can readily be applied to downstream tasks such as pick and place and
social navigation.
6552f3272056e27018cb1101
Matthew Chang
6552f3272056e27018cb1102
Theophile Gervet
admin_assigned
2023-11-14T10:45:03.548Z
644e822cd6001776ed7700e4
/avatars/58a72e6bf8ede3cee3de54873231fd3e.svg
Theophile Gervet
user
TheophileGervet
6552f3272056e27018cb1103
Mukul Khanna
admin_assigned
2023-11-14T10:45:10.582Z
6387a0328b3c55fea3e4a13d
https://cdn-avatars.huggingface.co/v1/production/uploads/6387a0328b3c55fea3e4a13d/uxMoiqxD0GLKV95ItECJr.jpeg
Mukul Khanna
user
mukulkhanna
6552f3272056e27018cb1104
Sriram Yenamandra
6552f3272056e27018cb1105
Dhruv Shah
claimed_verified
2023-11-15T09:21:06.183Z
630478649aef62c4013bbab2
/avatars/23a421be9fb37c0c6446e99adb618d37.svg
Dhruv
user
prieuredesion
6552f3272056e27018cb1106
So Yeon Min
6552f3272056e27018cb1107
Kavit Shah
admin_assigned
2023-11-14T10:46:14.695Z
649c9efa0c65a5d9697917d4
/avatars/4c66e4aa278be7709a8b729044583d5d.svg
Kavit Shah
user
KShah1998
6552f3272056e27018cb1108
Chris Paxton
admin_assigned
2023-11-14T10:46:21.416Z
62b13b28b7053ab38d2d5f96
https://cdn-avatars.huggingface.co/v1/production/uploads/62b13b28b7053ab38d2d5f96/V6780Omima7sNW5E4TEeW.png
Chris Paxton
user
cpaxton
6552f3272056e27018cb1109
Saurabh Gupta
claimed_verified
2023-11-15T09:21:02.788Z
65539ce59075e3304ee69344
/avatars/af3260f1398477a7d972dc034052bb3b.svg
Saurabh Gupta
user
s-gupta
6552f3272056e27018cb110a
Dhruv Batra
admin_assigned
2023-11-14T10:46:59.622Z
63ed1e57f765928ceeacbe02
/avatars/e5a7428bb0acb43a240f1c4a8c1a5639.svg
Dhruv Batra
user
dbatra
6552f3272056e27018cb110b
Roozbeh Mottaghi
6552f3272056e27018cb110c
Jitendra Malik
admin_assigned
2023-11-14T10:47:09.913Z
65369a95605a07338de78ab0
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/sGFjOjLT2akN-sn5beVWL.jpeg
Jitendra Malik 
user
jitendra1995
6552f3272056e27018cb110d
Devendra Singh Chaplot
admin_assigned
2023-11-14T10:47:21.169Z
65143e1c4f08b815c8db57a0
https://cdn-avatars.huggingface.co/v1/production/uploads/65143e1c4f08b815c8db57a0/JqkwKiJmLFRkH0NK3L8XH.jpeg
Devendra Singh Chaplot
user
devendrachaplot
GOAT: GO to Any Thing
In deployment scenarios such as homes and warehouses, mobile robots are
expected to autonomously navigate for extended periods, seamlessly executing
tasks articulated in terms that are intuitively understandable by human
operators. We present GO To Any Thing (GOAT), a universal navigation system
capable of tackling these requirements with three key features: a) Multimodal:
it can tackle goals specified via category labels, target images, and language
descriptions, b) Lifelong: it benefits from its past experience in the same
environment, and c) Platform Agnostic: it can be quickly deployed on robots
with different embodiments. GOAT is made possible through a modular system
design and a continually augmented instance-aware semantic memory that keeps
track of the appearance of objects from different viewpoints in addition to
category-level semantics. This enables GOAT to distinguish between different
instances of the same category to enable navigation to targets specified by
images and language descriptions. In experimental comparisons spanning over 90
hours in 9 different homes consisting of 675 goals selected across 200+
different object instances, we find GOAT achieves an overall success rate of
83%, surpassing previous methods and ablations by 32% (absolute improvement).
GOAT improves with experience in the environment, from a 60% success rate at
the first goal to a 90% success after exploration. In addition, we demonstrate
that GOAT can readily be applied to downstream tasks such as pick and place and
social navigation.
GOAT is a universal navigation system for mobile robots that uses multimodal input, learns from its experience, and is adaptable across different robotic platforms, achieving high success rates in navigation and other tasks.
multimodal lifelong learning platform agnostic instance-aware semantic memory category-level semantics navigation pick and place social navigation