Can Large Vision Language Models Read Maps Like a Human?
In this paper, we introduce MapBench-the first dataset specifically designed
for human-readable, pixel-based map-based outdoor navigation, curated from
complex path finding scenarios. MapBench comprises over 1600 pixel space map
path finding problems from 100 diverse maps. In MapBench, LVLMs generate
language-based navigation instructions given a map image and a query with
beginning and end landmarks. For each map, MapBench provides Map Space Scene
Graph (MSSG) as an indexing data structure to convert between natural language
and evaluate LVLM-generated results. We demonstrate that MapBench significantly
challenges state-of-the-art LVLMs both zero-shot prompting and a
Chain-of-Thought (CoT) augmented reasoning framework that decomposes map
navigation into sequential cognitive processes. Our evaluation of both
open-source and closed-source LVLMs underscores the substantial difficulty
posed by MapBench, revealing critical limitations in their spatial reasoning
and structured decision-making capabilities. We release all the code and
dataset in https://github.com/taco-group/MapBench.
67e192ae359a7a8b5fb32877
Shuo Xing
claimed_verified
2025-03-25T08:21:08.385Z
62df9ee3c7e6f74fc92a1351
/avatars/2a4f60ecc2fc4be2a8ac21c3f2ceb3cc.svg
Shuo Xing
user
shuoxing
67e192ae359a7a8b5fb32878
Zezhou Sun
67e192ae359a7a8b5fb32879
Shuangyu Xie
67e192ae359a7a8b5fb3287a
Kaiyuan Chen
claimed_verified
2025-05-22T07:18:40.366Z
66d2af23f040611f7cea1b1b
https://cdn-avatars.huggingface.co/v1/production/uploads/66d2af23f040611f7cea1b1b/vCToO_XgrLD0nfR8rzjZd.jpeg
Kaiyuan Eric Chen
user
keplerccc
67e192ae359a7a8b5fb3287b
Yanjia Huang
67e192ae359a7a8b5fb3287c
Yuping Wang
67e192ae359a7a8b5fb3287d
Jiachen Li
67e192ae359a7a8b5fb3287e
Dezhen Song
67e192ae359a7a8b5fb3287f
Zhengzhong Tu
claimed_verified
2025-06-03T08:55:25.632Z
62548d5fef3debb2ddf91217
/avatars/14975b45568f9c399c92c3986b6ce83e.svg
Zhengzhong Tu
user
vztu
Can Large Vision Language Models Read Maps Like a Human?
In this paper, we introduce MapBench-the first dataset specifically designed
for human-readable, pixel-based map-based outdoor navigation, curated from
complex path finding scenarios. MapBench comprises over 1600 pixel space map
path finding problems from 100 diverse maps. In MapBench, LVLMs generate
language-based navigation instructions given a map image and a query with
beginning and end landmarks. For each map, MapBench provides Map Space Scene
Graph (MSSG) as an indexing data structure to convert between natural language
and evaluate LVLM-generated results. We demonstrate that MapBench significantly
challenges state-of-the-art LVLMs both zero-shot prompting and a
Chain-of-Thought (CoT) augmented reasoning framework that decomposes map
navigation into sequential cognitive processes. Our evaluation of both
open-source and closed-source LVLMs underscores the substantial difficulty
posed by MapBench, revealing critical limitations in their spatial reasoning
and structured decision-making capabilities. We release all the code and
dataset in https://github.com/taco-group/MapBench.
MapBench challenges LVLMs with pixel-based map navigation problems, highlighting limitations in spatial reasoning and decision-making.
LVLMs Map Space Scene Graph (MSSG) zero-shot prompting Chain-of-Thought (CoT) spatial reasoning structured decision-making