UniGoal: Towards Universal Zero-shot Goal-oriented Navigation
In this paper, we propose a general framework for universal zero-shot
goal-oriented navigation. Existing zero-shot methods build inference framework
upon large language models (LLM) for specific tasks, which differs a lot in
overall pipeline and fails to generalize across different types of goal.
Towards the aim of universal zero-shot navigation, we propose a uniform graph
representation to unify different goals, including object category, instance
image and text description. We also convert the observation of agent into an
online maintained scene graph. With this consistent scene and goal
representation, we preserve most structural information compared with pure text
and are able to leverage LLM for explicit graph-based reasoning. Specifically,
we conduct graph matching between the scene graph and goal graph at each time
instant and propose different strategies to generate long-term goal of
exploration according to different matching states. The agent first iteratively
searches subgraph of goal when zero-matched. With partial matching, the agent
then utilizes coordinate projection and anchor pair alignment to infer the goal
location. Finally scene graph correction and goal verification are applied for
perfect matching. We also present a blacklist mechanism to enable robust switch
between stages. Extensive experiments on several benchmarks show that our
UniGoal achieves state-of-the-art zero-shot performance on three studied
navigation tasks with a single model, even outperforming task-specific
zero-shot methods and supervised universal methods.
67d39f4828221b583a33be2c
Hang Yin
admin_assigned
2025-03-14T11:05:49.754Z
6651034c05fb627b10f8abe5
/avatars/4fc0835d5956744295e515f8ff83b3df.svg
Hang Yin
user
hangyin
67d39f4828221b583a33be2d
Xiuwei Xu
admin_assigned
2025-03-14T11:05:43.448Z
648ac65fd044b25978015634
/avatars/2278a66fdc953220e9f8fc0ccce3ff00.svg
Xiuwei Xu
user
xuxw98
67d39f4828221b583a33be2e
Lingqing Zhao
67d39f4828221b583a33be2f
Ziwei Wang
67d39f4828221b583a33be30
Jie Zhou
67d39f4828221b583a33be31
Jiwen Lu
admin_assigned
2025-03-14T11:04:42.455Z
66c44203ea476bea05e9fcd7
/avatars/b061eebec609446e669f5ad6365959f9.svg
lu
user
jiwenlu
UniGoal: Towards Universal Zero-shot Goal-oriented Navigation
In this paper, we propose a general framework for universal zero-shot
goal-oriented navigation. Existing zero-shot methods build inference framework
upon large language models (LLM) for specific tasks, which differs a lot in
overall pipeline and fails to generalize across different types of goal.
Towards the aim of universal zero-shot navigation, we propose a uniform graph
representation to unify different goals, including object category, instance
image and text description. We also convert the observation of agent into an
online maintained scene graph. With this consistent scene and goal
representation, we preserve most structural information compared with pure text
and are able to leverage LLM for explicit graph-based reasoning. Specifically,
we conduct graph matching between the scene graph and goal graph at each time
instant and propose different strategies to generate long-term goal of
exploration according to different matching states. The agent first iteratively
searches subgraph of goal when zero-matched. With partial matching, the agent
then utilizes coordinate projection and anchor pair alignment to infer the goal
location. Finally scene graph correction and goal verification are applied for
perfect matching. We also present a blacklist mechanism to enable robust switch
between stages. Extensive experiments on several benchmarks show that our
UniGoal achieves state-of-the-art zero-shot performance on three studied
navigation tasks with a single model, even outperforming task-specific
zero-shot methods and supervised universal methods.
A universal zero-shot navigation framework using graph representations and large language models outperforms existing methods on various tasks.
zero-shot goal-oriented navigation large language models (LLM) uniform graph representation scene graph graph matching subgraph search coordinate projection anchor pair alignment scene graph correction goal verification blacklist mechanism