SpaceVista: All-Scale Visual Spatial Reasoning from mm to km
With the current surge in spatial reasoning explorations, researchers have
made significant progress in understanding indoor scenes, but still struggle
with diverse applications such as robotics and autonomous driving. This paper
aims to advance all-scale spatial reasoning across diverse scenarios by
tackling two key challenges: 1) the heavy reliance on indoor 3D scans and
labor-intensive manual annotations for dataset curation; 2) the absence of
effective all-scale scene modeling, which often leads to overfitting to
individual scenes. In this paper, we introduce a holistic solution that
integrates a structured spatial reasoning knowledge system, scale-aware
modeling, and a progressive training paradigm, as the first attempt to broaden
the all-scale spatial intelligence of MLLMs to the best of our knowledge. Using
a task-specific, specialist-driven automated pipeline, we curate over 38K video
scenes across 5 spatial scales to create SpaceVista-1M, a dataset comprising
approximately 1M spatial QA pairs spanning 19 diverse task types. While
specialist models can inject useful domain knowledge, they are not reliable for
evaluation. We then build an all-scale benchmark with precise annotations by
manually recording, retrieving, and assembling video-based data. However, naive
training with SpaceVista-1M often yields suboptimal results due to the
potential knowledge conflict. Accordingly, we introduce SpaceVista-7B, a
spatial reasoning model that accepts dense inputs beyond semantics and uses
scale as an anchor for scale-aware experts and progressive rewards. Finally,
extensive evaluations across 5 benchmarks, including our SpaceVista-Bench,
demonstrate competitive performance, showcasing strong generalization across
all scales and scenarios. Our dataset, model, and benchmark will be released on
https://peiwensun2000.github.io/mm2km .
68ec5975cd07fb414898c8ea
Peiwen Sun
claimed_verified
2025-10-13T10:06:05.983Z
667a518d58120f1b6ac579e8
/avatars/3e7d0e3d1e659ec29c0fca3e79df798e.svg
Peiwen Sun
user
spw2000
68ec5975cd07fb414898c8eb
Shiqiang Lang
68ec5975cd07fb414898c8ec
Dongming Wu
68ec5975cd07fb414898c8ed
Yi Ding
68ec5975cd07fb414898c8ee
Kaituo Feng
68ec5975cd07fb414898c8ef
Huadai Liu
68ec5975cd07fb414898c8f0
Zhen Ye
68ec5975cd07fb414898c8f1
Rui Liu
68ec5975cd07fb414898c8f2
Yun-Hui Liu
68ec5975cd07fb414898c8f3
Jianan Wang
68ec5975cd07fb414898c8f4
Xiangyu Yue
SpaceVista: All-Scale Visual Spatial Reasoning from mm to km
With the current surge in spatial reasoning explorations, researchers have
made significant progress in understanding indoor scenes, but still struggle
with diverse applications such as robotics and autonomous driving. This paper
aims to advance all-scale spatial reasoning across diverse scenarios by
tackling two key challenges: 1) the heavy reliance on indoor 3D scans and
labor-intensive manual annotations for dataset curation; 2) the absence of
effective all-scale scene modeling, which often leads to overfitting to
individual scenes. In this paper, we introduce a holistic solution that
integrates a structured spatial reasoning knowledge system, scale-aware
modeling, and a progressive training paradigm, as the first attempt to broaden
the all-scale spatial intelligence of MLLMs to the best of our knowledge. Using
a task-specific, specialist-driven automated pipeline, we curate over 38K video
scenes across 5 spatial scales to create SpaceVista-1M, a dataset comprising
approximately 1M spatial QA pairs spanning 19 diverse task types. While
specialist models can inject useful domain knowledge, they are not reliable for
evaluation. We then build an all-scale benchmark with precise annotations by
manually recording, retrieving, and assembling video-based data. However, naive
training with SpaceVista-1M often yields suboptimal results due to the
potential knowledge conflict. Accordingly, we introduce SpaceVista-7B, a
spatial reasoning model that accepts dense inputs beyond semantics and uses
scale as an anchor for scale-aware experts and progressive rewards. Finally,
extensive evaluations across 5 benchmarks, including our SpaceVista-Bench,
demonstrate competitive performance, showcasing strong generalization across
all scales and scenarios. Our dataset, model, and benchmark will be released on
https://peiwensun2000.github.io/mm2km .
A spatial reasoning model using scale-aware experts and progressive rewards demonstrates competitive performance across diverse tasks and scales using a large, curated dataset.
structured spatial reasoning scale-aware modeling progressive training spatial QA pairs spatial reasoning model scale-aware experts progressive rewards SpaceVista-1M SpaceVista-7B SpaceVista-Bench