PurpCode-R1: Reasoning for Safer Code Generation
Jiawei Liu, Nirav Diwan, Zhe Wang, Haoyu Zhai, Xiaona Zhou, Kiet Nguyen, Tianjiao Yu, Muntasir Wahed, Yinlin Deng, Hadjer Benkraouda, Yuxiang Wei, LINGMING ZHANG, Ismini Lourentzou, Gang Wang
We introduce PurpCode-R1, a novel post-training recipe that aligns coding assistants to perform safety reasoning to defend against malicious cyber activities and provide secure and functional code. PurpCode-R1 trains a reasoning model in two stages: (i) Rule Learning, which explicitly teaches the model to reference cyber safety rules to avoid facilitating malicious cyber events and to generate vulnerability-free code; and (ii) Reinforcement Learning, which optimizes model utility and safety through multi-objective diverse reward mechanisms. To empower the training pipelines with safety data, we perform internal red-teaming to synthesize comprehensive and high-coverage prompts based on real-world tasks for inducing unsafe cyber activities in the model. Based on PurpCode-R1, we develop PurpCode-R1-32B, which outperforms existing SOTA models in cyber safety, while preserving utilities in code generation and basic security knowledge.