CodePlot-CoT: Mathematical Visual Reasoning by Thinking with Code-Driven
  Images
Recent advances in Large Language Models (LLMs) and Vision Language Models
(VLMs) have shown significant progress in mathematical reasoning, yet they
still face a critical bottleneck with problems requiring visual assistance,
such as drawing auxiliary lines or plotting functions to solve the problems.
Most LLMs and VLMs are constrained to text-only reasoning chains, while
multimodal unified models that can generate interleaved text and images lack
the necessary precision and controllability for such tasks. To address this, we
propose CodePlot-CoT, a code-driven Chain-of-Thought paradigm for "thinking
with images" in mathematics. Our approach leverages the VLM to generate text
reasoning as well as executable plotting code, which is then rendered into
images as "visual thought", to solve mathematical problems. To achieve this, we
first construct Math-VR, the first large-scale, bilingual dataset and benchmark
for Mathematics problems with Visual Reasoning, comprising 178K samples.
Second, to create high-quality training data, we develop a state-of-the-art
image-to-code converter specialized for parsing complex mathematical figures
into codes. Finally, using these training data, we train the CodePlot-CoT model
for solving mathematical problems. Experimental results show that our model
achieves up to 21% increase over base model on our new benchmark, fully
validating the efficacy of our proposed code-driven reasoning paradigm. Our
work opens a new direction for multimodal mathematical reasoning and provides
the community with the first large-scale dataset, comprehensive benchmark, and
strong approach for such problems. To facilitate future research, we make our
datasets, code, and pretrained models publicly available at
https://github.com/HKU-MMLab/Math-VR-CodePlot-CoT.
68edb609de1fee572713a78f
Chengqi Duan
claimed_verified
2025-10-15T15:25:18.378Z
64a2b496e2e19de17db7de65
/avatars/241448ca487833d6cc5d57bb1fdb6ee5.svg
Duan Chengqi
user
gogoduan
68edb609de1fee572713a790
Kaiyue Sun
claimed_verified
2025-10-15T15:25:20.759Z
63640ce5ff4b318d1b7b6f5c
/avatars/9336d1dab1491f3c8e84b1ea287eb891.svg
Kaiyue Sun
user
Kaiyue
68edb609de1fee572713a791
Rongyao Fang
68edb609de1fee572713a792
Manyuan Zhang
68edb609de1fee572713a793
Yan Feng
68edb609de1fee572713a794
Ying Luo
68edb609de1fee572713a795
Yufang Liu
68edb609de1fee572713a796
Ke Wang
claimed_verified
2025-10-14T10:23:46.766Z
64d592c28767727dffa1f002
/avatars/fe38bcac944a2742dc12c624e62d24ef.svg
WangKe
user
scikkk
68edb609de1fee572713a797
Peng Pei
68edb609de1fee572713a798
Xunliang Cai
68edb609de1fee572713a799
Hongsheng Li
68edb609de1fee572713a79a
Yi Ma
68edb609de1fee572713a79b
Xihui Liu
claimed_verified
2025-10-14T07:30:42.570Z
65d5ec74cd05bc1eaa125040
/avatars/2de1b1539a86452c2c89570eeb02f5ab.svg
Xihui Liu
user
XihuiLiu
CodePlot-CoT: Mathematical Visual Reasoning by Thinking with Code-Driven
  Images
Recent advances in Large Language Models (LLMs) and Vision Language Models
(VLMs) have shown significant progress in mathematical reasoning, yet they
still face a critical bottleneck with problems requiring visual assistance,
such as drawing auxiliary lines or plotting functions to solve the problems.
Most LLMs and VLMs are constrained to text-only reasoning chains, while
multimodal unified models that can generate interleaved text and images lack
the necessary precision and controllability for such tasks. To address this, we
propose CodePlot-CoT, a code-driven Chain-of-Thought paradigm for "thinking
with images" in mathematics. Our approach leverages the VLM to generate text
reasoning as well as executable plotting code, which is then rendered into
images as "visual thought", to solve mathematical problems. To achieve this, we
first construct Math-VR, the first large-scale, bilingual dataset and benchmark
for Mathematics problems with Visual Reasoning, comprising 178K samples.
Second, to create high-quality training data, we develop a state-of-the-art
image-to-code converter specialized for parsing complex mathematical figures
into codes. Finally, using these training data, we train the CodePlot-CoT model
for solving mathematical problems. Experimental results show that our model
achieves up to 21% increase over base model on our new benchmark, fully
validating the efficacy of our proposed code-driven reasoning paradigm. Our
work opens a new direction for multimodal mathematical reasoning and provides
the community with the first large-scale dataset, comprehensive benchmark, and
strong approach for such problems. To facilitate future research, we make our
datasets, code, and pretrained models publicly available at
https://github.com/HKU-MMLab/Math-VR-CodePlot-CoT.
CodePlot-CoT, a code-driven Chain-of-Thought model, enhances multimodal mathematical reasoning by generating both text and executable plotting code to solve problems requiring visual assistance.
Large Language Models Vision Language Models mathematical reasoning visual assistance multimodal unified models Chain-of-Thought Math-VR image-to-code converter visual reasoning mathematical problems