Reasoning in Space via Grounding in the World
In this paper, we claim that 3D visual grounding is the cornerstone of
spatial reasoning and introduce the Grounded-Spatial Reasoner (GS-Reasoner) to
explore the effective spatial representations that bridge the gap between them.
Existing 3D LLMs suffer from the absence of a unified 3D representation capable
of jointly capturing semantic and geometric information. This deficiency is
manifested either in poor performance on grounding or in an excessive reliance
on external modules, ultimately hindering the seamless integration of grounding
and spatial reasoning. To address this, we propose a simple yet effective
dual-path pooling mechanism that tightly aligns geometric features with both
semantic and positional cues, constructing a unified image patch-based 3D
representation that encapsulates all essential information without increasing
the number of input tokens. Leveraging this holistic representation,
GS-Reasoner is the first 3D LLM that achieves autoregressive grounding entirely
without external modules while delivering performance comparable to
state-of-the-art models, establishing a unified and self-contained framework
for 3D spatial reasoning. To further bridge grounding and spatial reasoning, we
introduce the Grounded Chain-of-Thought (GCoT) dataset. This dataset is
meticulously curated to include both 3D bounding box annotations for objects
referenced in reasoning questions and step-by-step reasoning paths that
integrate grounding as a core component of the problem-solving process.
Extensive experiments demonstrate that GS-Reasoner achieves impressive results
on 3D visual grounding, which in turn significantly enhances its spatial
reasoning capabilities, leading to state-of-the-art performance.
68f09ba636f8b025381e1a7d
Yiming Chen
68f09ba636f8b025381e1a7e
Zekun Qi
claimed_verified
2025-10-20T12:13:35.828Z
63c3e8abc7d7f4c63a515a02
https://cdn-avatars.huggingface.co/v1/production/uploads/63c3e8abc7d7f4c63a515a02/npMHnVP2hHLbvoUGe7C4O.jpeg
Zekun Qi
user
qizekun
68f09ba636f8b025381e1a7f
Wenyao Zhang
68f09ba636f8b025381e1a80
Xin Jin
68f09ba636f8b025381e1a81
Li Zhang
68f09ba636f8b025381e1a82
Peidong Liu
Reasoning in Space via Grounding in the World
In this paper, we claim that 3D visual grounding is the cornerstone of
spatial reasoning and introduce the Grounded-Spatial Reasoner (GS-Reasoner) to
explore the effective spatial representations that bridge the gap between them.
Existing 3D LLMs suffer from the absence of a unified 3D representation capable
of jointly capturing semantic and geometric information. This deficiency is
manifested either in poor performance on grounding or in an excessive reliance
on external modules, ultimately hindering the seamless integration of grounding
and spatial reasoning. To address this, we propose a simple yet effective
dual-path pooling mechanism that tightly aligns geometric features with both
semantic and positional cues, constructing a unified image patch-based 3D
representation that encapsulates all essential information without increasing
the number of input tokens. Leveraging this holistic representation,
GS-Reasoner is the first 3D LLM that achieves autoregressive grounding entirely
without external modules while delivering performance comparable to
state-of-the-art models, establishing a unified and self-contained framework
for 3D spatial reasoning. To further bridge grounding and spatial reasoning, we
introduce the Grounded Chain-of-Thought (GCoT) dataset. This dataset is
meticulously curated to include both 3D bounding box annotations for objects
referenced in reasoning questions and step-by-step reasoning paths that
integrate grounding as a core component of the problem-solving process.
Extensive experiments demonstrate that GS-Reasoner achieves impressive results
on 3D visual grounding, which in turn significantly enhances its spatial
reasoning capabilities, leading to state-of-the-art performance.
GS-Reasoner, a 3D LLM with a dual-path pooling mechanism, achieves autoregressive grounding and state-of-the-art spatial reasoning without external modules.
3D visual grounding spatial reasoning Grounded-Spatial Reasoner (GS-Reasoner) 3D LLMs unified 3D representation dual-path pooling mechanism geometric features semantic cues positional cues image patch-based representation autoregressive grounding Grounded Chain-of-Thought (GCoT) dataset 3D bounding box annotations step-by-step reasoning paths