CodeI/O: Condensing Reasoning Patterns via Code Input-Output Prediction
Reasoning is a fundamental capability of Large Language Models. While prior
research predominantly focuses on enhancing narrow skills like math or code
generation, improving performance on many other reasoning tasks remains
challenging due to sparse and fragmented training data. To address this issue,
we propose CodeI/O, a novel approach that systematically condenses diverse
reasoning patterns inherently embedded in contextually-grounded codes, through
transforming the original code into a code input-output prediction format. By
training models to predict inputs/outputs given code and test cases entirely in
natural language as Chain-of-Thought (CoT) rationales, we expose them to
universal reasoning primitives -- like logic flow planning, state-space
searching, decision tree traversal, and modular decomposition -- while
decoupling structured reasoning from code-specific syntax and preserving
procedural rigor. Experimental results demonstrate CodeI/O leads to consistent
improvements across symbolic, scientific, logic, math & numerical, and
commonsense reasoning tasks. By matching the existing ground-truth outputs or
re-executing the code with predicted inputs, we can verify each prediction and
further enhance the CoTs through multi-turn revision, resulting in CodeI/O++
and achieving higher performance. Our data and models are available at
https://github.com/hkust-nlp/CodeIO.
67ac0ab720e98bddc5c19fed
Junlong Li
67ac0ab720e98bddc5c19fee
Daya Guo
67ac0ab720e98bddc5c19fef
Dejian Yang
67ac0ab720e98bddc5c19ff0
Runxin Xu
67ac0ab720e98bddc5c19ff1
Yu Wu
67ac0ab720e98bddc5c19ff2
Junxian He
CodeI/O: Condensing Reasoning Patterns via Code Input-Output Prediction
Reasoning is a fundamental capability of Large Language Models. While prior
research predominantly focuses on enhancing narrow skills like math or code
generation, improving performance on many other reasoning tasks remains
challenging due to sparse and fragmented training data. To address this issue,
we propose CodeI/O, a novel approach that systematically condenses diverse
reasoning patterns inherently embedded in contextually-grounded codes, through
transforming the original code into a code input-output prediction format. By
training models to predict inputs/outputs given code and test cases entirely in
natural language as Chain-of-Thought (CoT) rationales, we expose them to
universal reasoning primitives -- like logic flow planning, state-space
searching, decision tree traversal, and modular decomposition -- while
decoupling structured reasoning from code-specific syntax and preserving
procedural rigor. Experimental results demonstrate CodeI/O leads to consistent
improvements across symbolic, scientific, logic, math & numerical, and
commonsense reasoning tasks. By matching the existing ground-truth outputs or
re-executing the code with predicted inputs, we can verify each prediction and
further enhance the CoTs through multi-turn revision, resulting in CodeI/O++
and achieving higher performance. Our data and models are available at
https://github.com/hkust-nlp/CodeIO.
CodeI/O transforms code into a code input-output prediction format to train models on universal reasoning primitives and improve performance across various reasoning tasks through CoT rationales and multi-turn revision.
CodeI/O Chain-of-Thought logic flow planning state-space searching decision tree traversal modular decomposition CoT rationales multi-turn revision CodeI/O++