Thinking with Images for Multimodal Reasoning: Foundations, Methods, and
  Future Frontiers
Recent progress in multimodal reasoning has been significantly advanced by
textual Chain-of-Thought (CoT), a paradigm where models conduct reasoning
within language. This text-centric approach, however, treats vision as a
static, initial context, creating a fundamental "semantic gap" between rich
perceptual data and discrete symbolic thought. Human cognition often transcends
language, utilizing vision as a dynamic mental sketchpad. A similar evolution
is now unfolding in AI, marking a fundamental paradigm shift from models that
merely think about images to those that can truly think with images. This
emerging paradigm is characterized by models leveraging visual information as
intermediate steps in their thought process, transforming vision from a passive
input into a dynamic, manipulable cognitive workspace. In this survey, we chart
this evolution of intelligence along a trajectory of increasing cognitive
autonomy, which unfolds across three key stages: from external tool
exploration, through programmatic manipulation, to intrinsic imagination. To
structure this rapidly evolving field, our survey makes four key contributions.
(1) We establish the foundational principles of the think with image paradigm
and its three-stage framework. (2) We provide a comprehensive review of the
core methods that characterize each stage of this roadmap. (3) We analyze the
critical landscape of evaluation benchmarks and transformative applications.
(4) We identify significant challenges and outline promising future directions.
By providing this structured overview, we aim to offer a clear roadmap for
future research towards more powerful and human-aligned multimodal AI.
68674e689db35afc9c304d4c
Zhaochen Su
claimed_verified
2025-07-05T07:53:14.414Z
64264095ba51f8a2136946a0
https://cdn-avatars.huggingface.co/v1/production/uploads/64264095ba51f8a2136946a0/FR33boVpkDXcrvGMBmprF.jpeg
Zhaochen Su
user
Warrieryes
68674e689db35afc9c304d4d
Peng Xia
claimed_verified
2025-07-07T11:24:55.247Z
643e9ee6f6bb3c31a26e7bc4
/avatars/acfaa7d6a23dada24c86b954c3be116a.svg
Peng Xia
user
richardxp888
68674e689db35afc9c304d4e
Hangyu Guo
68674e689db35afc9c304d4f
Zhenhua Liu
68674e689db35afc9c304d50
Yan Ma
claimed_verified
2025-07-05T07:53:12.509Z
633fc70529b5a95f6e15a6b7
https://cdn-avatars.huggingface.co/v1/production/uploads/633fc70529b5a95f6e15a6b7/Fzh7wWuqU-fBbzdupOUtF.jpeg
Yan Ma
user
ManTle
68674e689db35afc9c304d51
Xiaoye Qu
claimed_verified
2025-07-04T07:48:48.311Z
64cb54da1af278541d663708
/avatars/c44507cc92bb2e83154bad31b90ce6dd.svg
Xiaoye Qu
user
Xiaoye08
68674e689db35afc9c304d52
Jiaqi Liu
claimed_verified
2025-07-07T11:24:53.066Z
684ff37fa383bc5d6b0ff77f
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/0JPr-cd_rxQz3k6rmzBOF.png
JiaqiLiu
user
JiaaqiLiu
68674e689db35afc9c304d53
Yanshu Li
68674e689db35afc9c304d54
Kaide Zeng
68674e689db35afc9c304d55
Zhengyuan Yang
68674e689db35afc9c304d56
Linjie Li
68674e689db35afc9c304d57
Yu Cheng
68674e689db35afc9c304d58
Heng Ji
68674e689db35afc9c304d59
Junxian He
68674e689db35afc9c304d5a
Yi R. Fung
Thinking with Images for Multimodal Reasoning: Foundations, Methods, and
  Future Frontiers
Recent progress in multimodal reasoning has been significantly advanced by
textual Chain-of-Thought (CoT), a paradigm where models conduct reasoning
within language. This text-centric approach, however, treats vision as a
static, initial context, creating a fundamental "semantic gap" between rich
perceptual data and discrete symbolic thought. Human cognition often transcends
language, utilizing vision as a dynamic mental sketchpad. A similar evolution
is now unfolding in AI, marking a fundamental paradigm shift from models that
merely think about images to those that can truly think with images. This
emerging paradigm is characterized by models leveraging visual information as
intermediate steps in their thought process, transforming vision from a passive
input into a dynamic, manipulable cognitive workspace. In this survey, we chart
this evolution of intelligence along a trajectory of increasing cognitive
autonomy, which unfolds across three key stages: from external tool
exploration, through programmatic manipulation, to intrinsic imagination. To
structure this rapidly evolving field, our survey makes four key contributions.
(1) We establish the foundational principles of the think with image paradigm
and its three-stage framework. (2) We provide a comprehensive review of the
core methods that characterize each stage of this roadmap. (3) We analyze the
critical landscape of evaluation benchmarks and transformative applications.
(4) We identify significant challenges and outline promising future directions.
By providing this structured overview, we aim to offer a clear roadmap for
future research towards more powerful and human-aligned multimodal AI.
The survey outlines the evolution of multimodal AI from treating vision as static context to integrating it dynamically into the reasoning process, highlighting three stages and key challenges.
multimodal reasoning Chain-of-Thought (CoT) semantic gap dynamic mental sketchpad cognitive autonomy external tool exploration programmatic manipulation intrinsic imagination evaluation benchmarks transformative applications