MindJourney: Test-Time Scaling with World Models for Spatial Reasoning
Spatial reasoning in 3D space is central to human cognition and indispensable
for embodied tasks such as navigation and manipulation. However,
state-of-the-art vision-language models (VLMs) struggle frequently with tasks
as simple as anticipating how a scene will look after an egocentric motion:
they perceive 2D images but lack an internal model of 3D dynamics. We therefore
propose MindJourney, a test-time scaling framework that grants a VLM with this
missing capability by coupling it to a controllable world model based on video
diffusion. The VLM iteratively sketches a concise camera trajectory, while the
world model synthesizes the corresponding view at each step. The VLM then
reasons over this multi-view evidence gathered during the interactive
exploration. Without any fine-tuning, our MindJourney achieves over an average
8% performance boost on the representative spatial reasoning benchmark SAT,
showing that pairing VLMs with world models for test-time scaling offers a
simple, plug-and-play route to robust 3D reasoning. Meanwhile, our method also
improves upon the test-time inference VLMs trained through reinforcement
learning, which demonstrates the potential of our method that utilizes world
models for test-time scaling.
6879af4f21b37e676c8e409b
Yuncong Yang
claimed_verified
2025-07-18T07:49:42.036Z
65e919332fd9300c7eb96556
/avatars/a826f7e14acf34603aa68e4fc27f12af.svg
Yuncong Yang
user
yyuncong
6879af4f21b37e676c8e409c
Jiageng Liu
6879af4f21b37e676c8e409d
Zheyuan Zhang
6879af4f21b37e676c8e409e
Siyuan Zhou
6879af4f21b37e676c8e409f
Reuben Tan
6879af4f21b37e676c8e40a0
Jianwei Yang
6879af4f21b37e676c8e40a1
Yilun Du
6879af4f21b37e676c8e40a2
Chuang Gan
MindJourney: Test-Time Scaling with World Models for Spatial Reasoning
Spatial reasoning in 3D space is central to human cognition and indispensable
for embodied tasks such as navigation and manipulation. However,
state-of-the-art vision-language models (VLMs) struggle frequently with tasks
as simple as anticipating how a scene will look after an egocentric motion:
they perceive 2D images but lack an internal model of 3D dynamics. We therefore
propose MindJourney, a test-time scaling framework that grants a VLM with this
missing capability by coupling it to a controllable world model based on video
diffusion. The VLM iteratively sketches a concise camera trajectory, while the
world model synthesizes the corresponding view at each step. The VLM then
reasons over this multi-view evidence gathered during the interactive
exploration. Without any fine-tuning, our MindJourney achieves over an average
8% performance boost on the representative spatial reasoning benchmark SAT,
showing that pairing VLMs with world models for test-time scaling offers a
simple, plug-and-play route to robust 3D reasoning. Meanwhile, our method also
improves upon the test-time inference VLMs trained through reinforcement
learning, which demonstrates the potential of our method that utilizes world
models for test-time scaling.
MindJourney enhances vision-language models with 3D reasoning by coupling them with a video diffusion-based world model, achieving improved performance on spatial reasoning tasks without fine-tuning.
vision-language models VLMs world model video diffusion camera trajectory multi-view evidence spatial reasoning SAT benchmark reinforcement learning