SSR: Enhancing Depth Perception in Vision-Language Models via
  Rationale-Guided Spatial Reasoning
Despite impressive advancements in Visual-Language Models (VLMs) for
multi-modal tasks, their reliance on RGB inputs limits precise spatial
understanding. Existing methods for integrating spatial cues, such as point
clouds or depth, either require specialized sensors or fail to effectively
exploit depth information for higher-order reasoning. To this end, we propose a
novel Spatial Sense and Reasoning method, dubbed SSR, a novel framework that
transforms raw depth data into structured, interpretable textual rationales.
These textual rationales serve as meaningful intermediate representations to
significantly enhance spatial reasoning capabilities. Additionally, we leverage
knowledge distillation to compress the generated rationales into compact latent
embeddings, which facilitate resource-efficient and plug-and-play integration
into existing VLMs without retraining. To enable comprehensive evaluation, we
introduce a new dataset named SSR-CoT, a million-scale visual-language
reasoning dataset enriched with intermediate spatial reasoning annotations, and
present SSRBench, a comprehensive multi-task benchmark. Extensive experiments
on multiple benchmarks demonstrate SSR substantially improves depth utilization
and enhances spatial reasoning, thereby advancing VLMs toward more human-like
multi-modal understanding. Our project page is at
https://yliu-cs.github.io/SSR.
682d47aa64daf8623f1f5604
Yang Liu
claimed_verified
2025-05-21T08:40:37.342Z
6586817e509bcae23f3dfc60
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/pwEStoey1XJYDi3F0Z930.png
Yang Liu
user
yliu-cs
682d47aa64daf8623f1f5605
Ming Ma
682d47aa64daf8623f1f5606
Xiaomin Yu
claimed_verified
2025-05-21T15:17:46.958Z
64084fa192033c150738e4f2
/avatars/dfff2216eb235c635e5abe6fda3084f0.svg
Yu_xm
user
Yu2020
682d47aa64daf8623f1f5607
Pengxiang Ding
682d47aa64daf8623f1f5608
Han Zhao
682d47aa64daf8623f1f5609
Mingyang Sun
682d47aa64daf8623f1f560a
Siteng Huang
682d47aa64daf8623f1f560b
Donglin Wang
SSR: Enhancing Depth Perception in Vision-Language Models via
  Rationale-Guided Spatial Reasoning
Despite impressive advancements in Visual-Language Models (VLMs) for
multi-modal tasks, their reliance on RGB inputs limits precise spatial
understanding. Existing methods for integrating spatial cues, such as point
clouds or depth, either require specialized sensors or fail to effectively
exploit depth information for higher-order reasoning. To this end, we propose a
novel Spatial Sense and Reasoning method, dubbed SSR, a novel framework that
transforms raw depth data into structured, interpretable textual rationales.
These textual rationales serve as meaningful intermediate representations to
significantly enhance spatial reasoning capabilities. Additionally, we leverage
knowledge distillation to compress the generated rationales into compact latent
embeddings, which facilitate resource-efficient and plug-and-play integration
into existing VLMs without retraining. To enable comprehensive evaluation, we
introduce a new dataset named SSR-CoT, a million-scale visual-language
reasoning dataset enriched with intermediate spatial reasoning annotations, and
present SSRBench, a comprehensive multi-task benchmark. Extensive experiments
on multiple benchmarks demonstrate SSR substantially improves depth utilization
and enhances spatial reasoning, thereby advancing VLMs toward more human-like
multi-modal understanding. Our project page is at
https://yliu-cs.github.io/SSR.
A novel method transforms depth data into textual rationales to enhance spatial reasoning in Visual-Language Models, improving depth utilization and human-like multi-modal understanding.
Spatial Sense and Reasoning SSR structured interpretable textual rationales knowledge distillation latent embeddings SSR-CoT SSRBench multi-task benchmark visual-language reasoning