OmniSpatial: Towards Comprehensive Spatial Reasoning Benchmark for
  Vision Language Models
Spatial reasoning is a key aspect of cognitive psychology and remains a major
bottleneck for current vision-language models (VLMs). While extensive research
has aimed to evaluate or improve VLMs' understanding of basic spatial
relations, such as distinguishing left from right, near from far, and object
counting, these tasks represent only the most fundamental level of spatial
reasoning. In this work, we introduce OmniSpatial, a comprehensive and
challenging benchmark for spatial reasoning, grounded in cognitive psychology.
OmniSpatial covers four major categories: dynamic reasoning, complex spatial
logic, spatial interaction, and perspective-taking, with 50 fine-grained
subcategories. Through Internet data crawling and careful manual annotation, we
construct over 1.5K question-answer pairs. Extensive experiments show that both
open- and closed-source VLMs, as well as existing reasoning and spatial
understanding models, exhibit significant limitations in comprehensive spatial
understanding. We further analyze failure cases and propose potential
directions for future research.
683fe55868402c738a8e5ee4
Mengdi Jia
claimed_verified
2025-06-05T07:28:05.023Z
6774dbb9607615a0ebfd21e9
https://cdn-avatars.huggingface.co/v1/production/uploads/6774dbb9607615a0ebfd21e9/L2JAcfpqjy7ixJidwy9zy.jpeg
Mengdi Jia
user
MengdiJia
683fe55868402c738a8e5ee5
Zekun Qi
claimed_verified
2025-06-04T08:54:22.042Z
63c3e8abc7d7f4c63a515a02
https://cdn-avatars.huggingface.co/v1/production/uploads/63c3e8abc7d7f4c63a515a02/npMHnVP2hHLbvoUGe7C4O.jpeg
Zekun Qi
user
qizekun
683fe55868402c738a8e5ee6
Shaochen Zhang
683fe55868402c738a8e5ee7
Wenyao Zhang
683fe55868402c738a8e5ee8
Xinqiang Yu
683fe55868402c738a8e5ee9
Jiawei He
683fe55868402c738a8e5eea
He Wang
683fe55868402c738a8e5eeb
Li Yi
OmniSpatial: Towards Comprehensive Spatial Reasoning Benchmark for
  Vision Language Models
Spatial reasoning is a key aspect of cognitive psychology and remains a major
bottleneck for current vision-language models (VLMs). While extensive research
has aimed to evaluate or improve VLMs' understanding of basic spatial
relations, such as distinguishing left from right, near from far, and object
counting, these tasks represent only the most fundamental level of spatial
reasoning. In this work, we introduce OmniSpatial, a comprehensive and
challenging benchmark for spatial reasoning, grounded in cognitive psychology.
OmniSpatial covers four major categories: dynamic reasoning, complex spatial
logic, spatial interaction, and perspective-taking, with 50 fine-grained
subcategories. Through Internet data crawling and careful manual annotation, we
construct over 1.5K question-answer pairs. Extensive experiments show that both
open- and closed-source VLMs, as well as existing reasoning and spatial
understanding models, exhibit significant limitations in comprehensive spatial
understanding. We further analyze failure cases and propose potential
directions for future research.
A comprehensive benchmark called OmniSpatial evaluates vision-language models' understanding of advanced spatial reasoning tasks, revealing significant limitations across various models.
vision-language models spatial reasoning cognitive psychology dynamic reasoning complex spatial logic spatial interaction perspective-taking question-answer pairs