CoRT: Code-integrated Reasoning within Thinking
Large Reasoning Models (LRMs) like o1 and DeepSeek-R1 have shown remarkable
progress in natural language reasoning with long chain-of-thought (CoT), yet
they remain inefficient or inaccurate when handling complex mathematical
operations. Addressing these limitations through computational tools (e.g.,
computation libraries and symbolic solvers) is promising, but it introduces a
technical challenge: Code Interpreter (CI) brings external knowledge beyond the
model's internal text representations, thus the direct combination is not
efficient. This paper introduces CoRT, a post-training framework for teaching
LRMs to leverage CI effectively and efficiently. As a first step, we address
the data scarcity issue by synthesizing code-integrated reasoning data through
Hint-Engineering, which strategically inserts different hints at appropriate
positions to optimize LRM-CI interaction. We manually create 30 high-quality
samples, upon which we post-train models ranging from 1.5B to 32B parameters,
with supervised fine-tuning, rejection fine-tuning and reinforcement learning.
Our experimental results demonstrate that Hint-Engineering models achieve 4\%
and 8\% absolute improvements on DeepSeek-R1-Distill-Qwen-32B and
DeepSeek-R1-Distill-Qwen-1.5B respectively, across five challenging
mathematical reasoning datasets. Furthermore, Hint-Engineering models use about
30\% fewer tokens for the 32B model and 50\% fewer tokens for the 1.5B model
compared with the natural language models. The models and code are available at
https://github.com/ChengpengLi1003/CoRT.
684ae213dbd21a9cc27b100b
Chengpeng Li
684ae213dbd21a9cc27b100c
Zhengyang Tang
claimed_verified
2025-06-13T07:41:11.984Z
64912976b95c3f0a1e6233cb
/avatars/c0615f8c6606073faffb419757d4e667.svg
Zhengyang Tang
user
tangzhy
684ae213dbd21a9cc27b100d
Ziniu Li
684ae213dbd21a9cc27b100e
Mingfeng Xue
684ae213dbd21a9cc27b100f
Keqin Bao
684ae213dbd21a9cc27b1010
Tian Ding
684ae213dbd21a9cc27b1011
Ruoyu Sun
684ae213dbd21a9cc27b1012
Benyou Wang
684ae213dbd21a9cc27b1013
Xiang Wang
684ae213dbd21a9cc27b1014
Junyang Lin
684ae213dbd21a9cc27b1015
Dayiheng Liu
CoRT: Code-integrated Reasoning within Thinking
Large Reasoning Models (LRMs) like o1 and DeepSeek-R1 have shown remarkable
progress in natural language reasoning with long chain-of-thought (CoT), yet
they remain inefficient or inaccurate when handling complex mathematical
operations. Addressing these limitations through computational tools (e.g.,
computation libraries and symbolic solvers) is promising, but it introduces a
technical challenge: Code Interpreter (CI) brings external knowledge beyond the
model's internal text representations, thus the direct combination is not
efficient. This paper introduces CoRT, a post-training framework for teaching
LRMs to leverage CI effectively and efficiently. As a first step, we address
the data scarcity issue by synthesizing code-integrated reasoning data through
Hint-Engineering, which strategically inserts different hints at appropriate
positions to optimize LRM-CI interaction. We manually create 30 high-quality
samples, upon which we post-train models ranging from 1.5B to 32B parameters,
with supervised fine-tuning, rejection fine-tuning and reinforcement learning.
Our experimental results demonstrate that Hint-Engineering models achieve 4\%
and 8\% absolute improvements on DeepSeek-R1-Distill-Qwen-32B and
DeepSeek-R1-Distill-Qwen-1.5B respectively, across five challenging
mathematical reasoning datasets. Furthermore, Hint-Engineering models use about
30\% fewer tokens for the 32B model and 50\% fewer tokens for the 1.5B model
compared with the natural language models. The models and code are available at
https://github.com/ChengpengLi1003/CoRT.
CoRT enhances Large Reasoning Models with Code Interpreter using Hint-Engineering, improving mathematical reasoning efficiency and reducing token usage.
Large Reasoning Models LRMs DeepSeek-R1 chain-of-thought CoT complex mathematical operations Code Interpreter CI Hint-Engineering supervised fine-tuning rejection fine-tuning reinforcement learning mathematical reasoning datasets