CoRT: Code-integrated Reasoning within Thinking
Large Reasoning Models (LRMs) like o1 and DeepSeek-R1 have shown remarkable
progress in natural language reasoning with long chain-of-thought (CoT), yet
they remain inefficient or inaccurate when handling complex mathematical
operations. Addressing these limitations through computational tools (e.g.,
computation libraries and symbolic solvers) is promising, but it introduces a
technical challenge: Code Interpreter (CI) brings external knowledge beyond the
model's internal text representations, thus the direct combination is not
efficient. This paper introduces CoRT, a post-training framework for teaching
LRMs to leverage CI effectively and efficiently. As a first step, we address
the data scarcity issue by synthesizing code-integrated reasoning data through
Hint-Engineering, which strategically inserts different hints at appropriate
positions to optimize LRM-CI interaction. We manually create 30 high-quality
samples, upon which we post-train models ranging from 1.5B to 32B parameters,
with supervised fine-tuning, rejection fine-tuning and reinforcement learning.
Our experimental results demonstrate that Hint-Engineering models achieve 4\%
and 8\% absolute improvements on DeepSeek-R1-Distill-Qwen-32B and
DeepSeek-R1-Distill-Qwen-1.5B respectively, across five challenging
mathematical reasoning datasets. Furthermore, Hint-Engineering models use about
30\% fewer tokens for the 32B model and 50\% fewer tokens for the 1.5B model
compared with the natural language models. The models and code are available at
https://github.com/ChengpengLi1003/CoRT.
684ae213dbd21a9cc27b100b
Chengpeng Li
684ae213dbd21a9cc27b100c
Zhengyang Tang
claimed_verified
2025-06-13T07:41:11.984Z
64912976b95c3f0a1e6233cb
/avatars/3e338c5eef2514055ed98ae6141a5d1a.svg
Zhengyang Tang
user
tangzhy
684ae213dbd21a9cc27b100d
Ziniu Li
claimed_verified
2025-11-03T21:03:13.358Z
647c4c901f878439e2fd34d6
/avatars/b4399d210d7239d4662b11a4ee7b527d.svg
Ziniu Li
user
ziniuli
684ae213dbd21a9cc27b100e
Mingfeng Xue
684ae213dbd21a9cc27b100f
Keqin Bao
684ae213dbd21a9cc27b1010
Tian Ding
684ae213dbd21a9cc27b1011
Ruoyu Sun
684ae213dbd21a9cc27b1012
Benyou Wang
684ae213dbd21a9cc27b1013
Xiang Wang
684ae213dbd21a9cc27b1014
Junyang Lin
684ae213dbd21a9cc27b1015
Dayiheng Liu
CoRT: Code-integrated Reasoning within Thinking
Large Reasoning Models (LRMs) like o1 and DeepSeek-R1 have shown remarkable
progress in natural language reasoning with long chain-of-thought (CoT), yet
they remain inefficient or inaccurate when handling complex mathematical
operations. Addressing these limitations through computational tools (e.g.,
computation libraries and symbolic solvers) is promising, but it introduces a
technical challenge: Code Interpreter (CI) brings external knowledge beyond the
model's internal text representations, thus the direct combination is not
efficient. This paper introduces CoRT, a post-training framework for teaching
LRMs to leverage CI effectively and efficiently. As a first step, we address
the data scarcity issue by synthesizing code-integrated reasoning data through
Hint-Engineering, which strategically inserts different hints at appropriate
positions to optimize LRM-CI interaction. We manually create 30 high-quality
samples, upon which we post-train models ranging from 1.5B to 32B parameters,
with supervised fine-tuning, rejection fine-tuning and reinforcement learning.
Our experimental results demonstrate that Hint-Engineering models achieve 4\%
and 8\% absolute improvements on DeepSeek-R1-Distill-Qwen-32B and
DeepSeek-R1-Distill-Qwen-1.5B respectively, across five challenging
mathematical reasoning datasets. Furthermore, Hint-Engineering models use about
30\% fewer tokens for the 32B model and 50\% fewer tokens for the 1.5B model
compared with the natural language models. The models and code are available at
https://github.com/ChengpengLi1003/CoRT.
CoRT enhances Large Reasoning Models with Code Interpreter using Hint-Engineering, improving mathematical reasoning efficiency and reducing token usage.
Large Reasoning Models LRMs DeepSeek-R1 chain-of-thought CoT complex mathematical operations Code Interpreter CI Hint-Engineering supervised fine-tuning rejection fine-tuning reinforcement learning mathematical reasoning datasets