Can Large Vision Language Models Read Maps Like a Human?
In this paper, we introduce MapBench-the first dataset specifically designed
for human-readable, pixel-based map-based outdoor navigation, curated from
complex path finding scenarios. MapBench comprises over 1600 pixel space map
path finding problems from 100 diverse maps. In MapBench, LVLMs generate
language-based navigation instructions given a map image and a query with
beginning and end landmarks. For each map, MapBench provides Map Space Scene
Graph (MSSG) as an indexing data structure to convert between natural language
and evaluate LVLM-generated results. We demonstrate that MapBench significantly
challenges state-of-the-art LVLMs both zero-shot prompting and a
Chain-of-Thought (CoT) augmented reasoning framework that decomposes map
navigation into sequential cognitive processes. Our evaluation of both
open-source and closed-source LVLMs underscores the substantial difficulty
posed by MapBench, revealing critical limitations in their spatial reasoning
and structured decision-making capabilities. We release all the code and
dataset in https://github.com/taco-group/MapBench.
67e192ae359a7a8b5fb32877
Shuo Xing
claimed_verified
2025-03-25T08:21:08.385Z
62df9ee3c7e6f74fc92a1351
/avatars/2a4f60ecc2fc4be2a8ac21c3f2ceb3cc.svg
Shuo Xing
user
shuoxing
67e192ae359a7a8b5fb32878
Zezhou Sun
67e192ae359a7a8b5fb32879
Shuangyu Xie
67e192ae359a7a8b5fb3287a
Kaiyuan Chen
67e192ae359a7a8b5fb3287b
Yanjia Huang
67e192ae359a7a8b5fb3287c
Yuping Wang
67e192ae359a7a8b5fb3287d
Jiachen Li
67e192ae359a7a8b5fb3287e
Dezhen Song
67e192ae359a7a8b5fb3287f
Zhengzhong Tu
Can Large Vision Language Models Read Maps Like a Human?
In this paper, we introduce MapBench-the first dataset specifically designed
for human-readable, pixel-based map-based outdoor navigation, curated from
complex path finding scenarios. MapBench comprises over 1600 pixel space map
path finding problems from 100 diverse maps. In MapBench, LVLMs generate
language-based navigation instructions given a map image and a query with
beginning and end landmarks. For each map, MapBench provides Map Space Scene
Graph (MSSG) as an indexing data structure to convert between natural language
and evaluate LVLM-generated results. We demonstrate that MapBench significantly
challenges state-of-the-art LVLMs both zero-shot prompting and a
Chain-of-Thought (CoT) augmented reasoning framework that decomposes map
navigation into sequential cognitive processes. Our evaluation of both
open-source and closed-source LVLMs underscores the substantial difficulty
posed by MapBench, revealing critical limitations in their spatial reasoning
and structured decision-making capabilities. We release all the code and
dataset in https://github.com/taco-group/MapBench.
MapBench LVLMs pixel-based map-based outdoor navigation pixel space map path finding problems Map Space Scene Graph (MSSG) language-based navigation instructions zero-shot prompting Chain-of-Thought (CoT) sequential cognitive processes spatial reasoning structured decision-making capabilities