Training Language Models to Generate Quality Code with Program Analysis
  Feedback
Code generation with large language models (LLMs), often termed vibe coding,
is increasingly adopted in production but fails to ensure code quality,
particularly in security (e.g., SQL injection vulnerabilities) and
maintainability (e.g., missing type annotations). Existing methods, such as
supervised fine-tuning and rule-based post-processing, rely on labor-intensive
annotations or brittle heuristics, limiting their scalability and
effectiveness. We propose REAL, a reinforcement learning framework that
incentivizes LLMs to generate production-quality code using program
analysis-guided feedback. Specifically, REAL integrates two automated signals:
(1) program analysis detecting security or maintainability defects and (2) unit
tests ensuring functional correctness. Unlike prior work, our framework is
prompt-agnostic and reference-free, enabling scalable supervision without
manual intervention. Experiments across multiple datasets and model scales
demonstrate that REAL outperforms state-of-the-art methods in simultaneous
assessments of functionality and code quality. Our work bridges the gap between
rapid prototyping and production-ready code, enabling LLMs to deliver both
speed and quality.
68409fbfeb249b555b0e4c60
Feng Yao
claimed_verified
2025-06-05T07:27:28.155Z
64c8b2c5c547ed5243e14a6e
/avatars/96d4a9010f96001c8cff235915926390.svg
Feng Yao
user
fengyao1909
68409fbfeb249b555b0e4c61
Zilong Wang
68409fbfeb249b555b0e4c62
Liyuan Liu
68409fbfeb249b555b0e4c63
Junxia Cui
68409fbfeb249b555b0e4c64
Li Zhong
68409fbfeb249b555b0e4c65
Xiaohan Fu
claimed_verified
2025-06-05T07:27:30.320Z
661ad1d2e7b0ab12bcabf85c
/avatars/7220799b6a676c30b5374d01f1370406.svg
Xiaohan Fu
user
x5fu
68409fbfeb249b555b0e4c66
Haohui Mai
68409fbfeb249b555b0e4c67
Vish Krishnan
68409fbfeb249b555b0e4c68
Jianfeng Gao
68409fbfeb249b555b0e4c69
Jingbo Shang
Training Language Models to Generate Quality Code with Program Analysis
  Feedback
Code generation with large language models (LLMs), often termed vibe coding,
is increasingly adopted in production but fails to ensure code quality,
particularly in security (e.g., SQL injection vulnerabilities) and
maintainability (e.g., missing type annotations). Existing methods, such as
supervised fine-tuning and rule-based post-processing, rely on labor-intensive
annotations or brittle heuristics, limiting their scalability and
effectiveness. We propose REAL, a reinforcement learning framework that
incentivizes LLMs to generate production-quality code using program
analysis-guided feedback. Specifically, REAL integrates two automated signals:
(1) program analysis detecting security or maintainability defects and (2) unit
tests ensuring functional correctness. Unlike prior work, our framework is
prompt-agnostic and reference-free, enabling scalable supervision without
manual intervention. Experiments across multiple datasets and model scales
demonstrate that REAL outperforms state-of-the-art methods in simultaneous
assessments of functionality and code quality. Our work bridges the gap between
rapid prototyping and production-ready code, enabling LLMs to deliver both
speed and quality.
A reinforcement learning framework improves code quality in large language models by using automated feedback from program analysis and unit tests.
reinforcement learning large language models code generation program analysis unit tests prompt-agnostic reference-free