Think with 3D: Geometric Imagination Grounded Spatial Reasoning from
  Limited Views
Though recent advances in vision-language models (VLMs) have achieved
remarkable progress across a wide range of multimodal tasks, understanding 3D
spatial relationships from limited views remains a significant challenge.
Previous reasoning methods typically rely on pure text (e.g., topological
cognitive maps) or on 2D visual cues. However, their limited representational
capacity hinders performance in specific tasks that require 3D spatial
imagination. To address this limitation, we propose 3DThinker, a framework that
can effectively exploits the rich geometric information embedded within images
while reasoning, like humans do. Our framework is the first to enable 3D
mentaling during reasoning without any 3D prior input, and it does not rely on
explicitly labeled 3D data for training. Specifically, our training consists of
two stages. First, we perform supervised training to align the 3D latent
generated by VLM while reasoning with that of a 3D foundation model (e.g.,
VGGT). Then, we optimize the entire reasoning trajectory solely based on
outcome signals, thereby refining the underlying 3D mentaling. Extensive
experiments across multiple benchmarks show that 3DThinker consistently
outperforms strong baselines and offers a new perspective toward unifying 3D
representations into multimodal reasoning. Our code will be available at
https://github.com/zhangquanchen/3DThinker.
68f8d62f6334fefb2d6023df
Zhangquan Chen
claimed_verified
2025-10-22T15:40:51.777Z
6458a99c3b81018d6b93aecb
/avatars/d90f16e8d3e49e095fea4cbd899837df.svg
jankin
user
jankin123
68f8d62f6334fefb2d6023e0
Manyuan Zhang
68f8d62f6334fefb2d6023e1
Xinlei Yu
68f8d62f6334fefb2d6023e2
Xufang Luo
68f8d62f6334fefb2d6023e3
Mingze Sun
68f8d62f6334fefb2d6023e4
Zihao Pan
claimed_verified
2025-10-22T15:40:54.539Z
65ad57da57f263e3d030187a
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/dJ3DYSIlv3Pb_6IEbqwOQ.png
潘子豪
user
Apostle723
68f8d62f6334fefb2d6023e5
Yan Feng
68f8d62f6334fefb2d6023e6
Peng Pei
68f8d62f6334fefb2d6023e7
Xunliang Cai
68f8d62f6334fefb2d6023e8
Ruqi Huang
Think with 3D: Geometric Imagination Grounded Spatial Reasoning from
  Limited Views
Though recent advances in vision-language models (VLMs) have achieved
remarkable progress across a wide range of multimodal tasks, understanding 3D
spatial relationships from limited views remains a significant challenge.
Previous reasoning methods typically rely on pure text (e.g., topological
cognitive maps) or on 2D visual cues. However, their limited representational
capacity hinders performance in specific tasks that require 3D spatial
imagination. To address this limitation, we propose 3DThinker, a framework that
can effectively exploits the rich geometric information embedded within images
while reasoning, like humans do. Our framework is the first to enable 3D
mentaling during reasoning without any 3D prior input, and it does not rely on
explicitly labeled 3D data for training. Specifically, our training consists of
two stages. First, we perform supervised training to align the 3D latent
generated by VLM while reasoning with that of a 3D foundation model (e.g.,
VGGT). Then, we optimize the entire reasoning trajectory solely based on
outcome signals, thereby refining the underlying 3D mentaling. Extensive
experiments across multiple benchmarks show that 3DThinker consistently
outperforms strong baselines and offers a new perspective toward unifying 3D
representations into multimodal reasoning. Our code will be available at
https://github.com/zhangquanchen/3DThinker.
3DThinker is a framework that enhances multimodal reasoning by integrating 3D spatial understanding from images without requiring 3D prior input or labeled data.
vision-language models 3D spatial relationships 3D mentaling 3D latent 3D foundation model VGGT multimodal reasoning