SAME: Learning Generic Language-Guided Visual Navigation with
  State-Adaptive Mixture of Experts
The academic field of learning instruction-guided visual navigation can be
generally categorized into high-level category-specific search and low-level
language-guided navigation, depending on the granularity of language
instruction, in which the former emphasizes the exploration process, while the
latter concentrates on following detailed textual commands. Despite the
differing focuses of these tasks, the underlying requirements of interpreting
instructions, comprehending the surroundings, and inferring action decisions
remain consistent. This paper consolidates diverse navigation tasks into a
unified and generic framework -- we investigate the core difficulties of
sharing general knowledge and exploiting task-specific capabilities in learning
navigation and propose a novel State-Adaptive Mixture of Experts (SAME) model
that effectively enables an agent to infer decisions based on
different-granularity language and dynamic observations. Powered by SAME, we
present a versatile agent capable of addressing seven navigation tasks
simultaneously that outperforms or achieves highly comparable performance to
task-specific agents.
675b9d8aa86e549855482d11
Gengze Zhou
claimed_verified
2024-12-16T09:42:27.297Z
6450bcd3673b2bcfaf8681af
/avatars/f5f93d780562d0772ec5dc1728945fcf.svg
Gengze Zhou
user
ZGZzz
675b9d8aa86e549855482d12
Yicong Hong
admin_assigned
2024-12-16T10:53:12.463Z
654ca66fe1671abcbc50a6e4
https://cdn-avatars.huggingface.co/v1/production/uploads/654ca66fe1671abcbc50a6e4/VqcnC17YcqC9Hk65_LxtE.jpeg
Yicong Hong
user
YicongHong
675b9d8aa86e549855482d13
Zun Wang
admin_assigned
2024-12-16T10:53:06.791Z
6622c710b0e5c5e3de8311c1
/avatars/a824c150040731679bbd77762ca9d4eb.svg
Zun Wang
user
ZunWang
675b9d8aa86e549855482d14
Chongyang Zhao
admin_assigned
2024-12-16T10:53:01.021Z
64c7671e77a6473ac325b6d8
/avatars/65434ae55207e785bd1ff2798d612fc2.svg
Chongyang Zhao
user
zhaoc5
675b9d8aa86e549855482d15
Mohit Bansal
admin_assigned
2024-12-16T10:52:54.647Z
665d9d3a057f7c508f98c625
https://cdn-avatars.huggingface.co/v1/production/uploads/665d9d3a057f7c508f98c625/u1R9P9sJoAl4zEIcetbPy.jpeg
Mohit Bansal
user
mohitbansal
675b9d8aa86e549855482d16
Qi Wu
SAME: Learning Generic Language-Guided Visual Navigation with
  State-Adaptive Mixture of Experts
The academic field of learning instruction-guided visual navigation can be
generally categorized into high-level category-specific search and low-level
language-guided navigation, depending on the granularity of language
instruction, in which the former emphasizes the exploration process, while the
latter concentrates on following detailed textual commands. Despite the
differing focuses of these tasks, the underlying requirements of interpreting
instructions, comprehending the surroundings, and inferring action decisions
remain consistent. This paper consolidates diverse navigation tasks into a
unified and generic framework -- we investigate the core difficulties of
sharing general knowledge and exploiting task-specific capabilities in learning
navigation and propose a novel State-Adaptive Mixture of Experts (SAME) model
that effectively enables an agent to infer decisions based on
different-granularity language and dynamic observations. Powered by SAME, we
present a versatile agent capable of addressing seven navigation tasks
simultaneously that outperforms or achieves highly comparable performance to
task-specific agents.
A State-Adaptive Mixture of Experts model enables a unified agent to perform multiple visual navigation tasks by adapting to different-granularity language and dynamic observations.
State-Adaptive Mixture of Experts SAME model visual navigation language-guided navigation category-specific search language instruction high-level search low-level navigation instruction interpretation action inference task-specific agents