Scaling Code-Assisted CoT for Model Reasoning
Honglin Lin, Qizhi Pei, Zhuoshi Pan, Yu Li, Xin Gao, Juntao Li, Lijun Wu, Conghui He
Reasoning capability is pivotal for large language models (LLMs) to solve complex tasks, yet achieving reliable and scalable reasoning remains challenging. While chain-of-thought (CoT) prompting has become a mainstream approach, existing methods often suffer from uncontrolled generation, insufficient quality, and limited diversity in reasoning paths. Recent efforts leverage code to enhance CoT by grounding reasoning in executable steps, but such methods typically rely on static code templates or human-curated rules, hindering scalability and generalizability. In this work, we propose Caco (Code-Assisted Chain-of-ThOught), a novel framework that automates the generation of high-quality, verifiable, and diverse reasoning data through code-driven augmentation. Unlike prior work, Caco first fine-tunes a code-based CoT generator on existing math and programming solutions in a unified code format, then scales the data generation to a large amount of diverse reasoning traces. Crucially, we introduce automated validation via code execution and rule-based filtering to ensure logical correctness and structural diversity, while reverse-engineering filtered outputs into natural language instructions to enrich task adaptability. This closed-loop process enables fully automated, scalable synthesis of reasoning data with guaranteed executability. Experiments on our created Caco-597 dataset demonstrate that Caco-trained models achieve state-of-the-art (SOTA) performance on mathematical reasoning benchmarks, outperforming existing strong baselines. Further analysis reveals that Caco’s code-anchored verification and instruction diversity contribute to superior generalization across unseen tasks. Our work establishes a paradigm for building self-sustaining, trustworthy reasoning systems without human intervention.