StepCoder: Improve Code Generation with Reinforcement Learning from
  Compiler Feedback
The advancement of large language models (LLMs) has significantly propelled
the field of code generation. Previous work integrated reinforcement learning
(RL) with compiler feedback for exploring the output space of LLMs to enhance
code generation quality. However, the lengthy code generated by LLMs in
response to complex human requirements makes RL exploration a challenge. Also,
since the unit tests may not cover the complicated code, optimizing LLMs by
using these unexecuted code snippets is ineffective. To tackle these
challenges, we introduce StepCoder, a novel RL framework for code generation,
consisting of two main components: CCCS addresses the exploration challenge by
breaking the long sequences code generation task into a Curriculum of Code
Completion Subtasks, while FGO only optimizes the model by masking the
unexecuted code segments to provide Fine-Grained Optimization. In addition, we
furthermore construct the APPS+ dataset for RL training, which is manually
verified to ensure the correctness of unit tests. Experimental results show
that our method improves the ability to explore the output space and
outperforms state-of-the-art approaches in corresponding benchmarks.
65c04ff5c63d6a8d7f234771
Shihan Dou
admin_assigned
2024-02-05T12:01:46.974Z
64ae0b6ccf90fe27556cf56e
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/uVXJd3TBssT_Ug4yAl3c9.jpeg
Shihan Dou
user
Ablustrund
65c04ff5c63d6a8d7f234772
Yan Liu
65c04ff5c63d6a8d7f234773
Haoxiang Jia
admin_assigned
2024-02-05T12:02:33.315Z
644606aee1fd8d65b27e2744
/avatars/08f742765f66114181a0be817c7fea5f.svg
Haoxiang Jia
user
jiatastic
65c04ff5c63d6a8d7f234774
Limao Xiong
65c04ff5c63d6a8d7f234775
Enyu Zhou
admin_assigned
2024-02-05T12:03:11.210Z
64c28861c0126fdf8a285c92
/avatars/409c914c827282f4276d3500561fabf3.svg
enyu
user
zhouenyu04
65c04ff5c63d6a8d7f234776
Junjie Shan
65c04ff5c63d6a8d7f234777
Caishuang Huang
65c04ff5c63d6a8d7f234778
Wei Shen
claimed_verified
2024-06-28T08:03:23.391Z
62308ae8be7833fa98620372
/avatars/220ed57376202c61ec1afbf5a4f45b67.svg
random
user
fakerbaby
65c04ff5c63d6a8d7f234779
Xiaoran Fan
admin_assigned
2024-02-05T12:03:47.870Z
6424fa5f988b8b19edba8ed4
/avatars/cff4f1535632e7c8abc33fe7b862f60d.svg
Xiaoran Fan
user
cnxup
65c04ff5c63d6a8d7f23477a
Zhiheng Xi
admin_assigned
2024-02-05T12:03:55.868Z
653a6e5cae155b92bae77b74
https://cdn-avatars.huggingface.co/v1/production/uploads/653a6e5cae155b92bae77b74/TA5FWKAUsB249ux4MzD_R.jpeg
Zhiheng Xi
user
WooooDyy
65c04ff5c63d6a8d7f23477b
Yuhao Zhou
admin_assigned
2024-02-05T12:04:03.901Z
646dbfcaf813cfe153ef2964
/avatars/8e73f91b35ac0d11671fd7399a43e723.svg
Yuhao Zhou
user
YuhaoZhou
65c04ff5c63d6a8d7f23477c
Tao Ji
65c04ff5c63d6a8d7f23477d
Rui Zheng
65c04ff5c63d6a8d7f23477e
Qi Zhang
65c04ff5c63d6a8d7f23477f
Xuanjing Huang
65c04ff5c63d6a8d7f234780
Tao Gui
StepCoder: Improve Code Generation with Reinforcement Learning from
  Compiler Feedback
The advancement of large language models (LLMs) has significantly propelled
the field of code generation. Previous work integrated reinforcement learning
(RL) with compiler feedback for exploring the output space of LLMs to enhance
code generation quality. However, the lengthy code generated by LLMs in
response to complex human requirements makes RL exploration a challenge. Also,
since the unit tests may not cover the complicated code, optimizing LLMs by
using these unexecuted code snippets is ineffective. To tackle these
challenges, we introduce StepCoder, a novel RL framework for code generation,
consisting of two main components: CCCS addresses the exploration challenge by
breaking the long sequences code generation task into a Curriculum of Code
Completion Subtasks, while FGO only optimizes the model by masking the
unexecuted code segments to provide Fine-Grained Optimization. In addition, we
furthermore construct the APPS+ dataset for RL training, which is manually
verified to ensure the correctness of unit tests. Experimental results show
that our method improves the ability to explore the output space and
outperforms state-of-the-art approaches in corresponding benchmarks.
StepCoder, a reinforcement learning framework, enhances code generation by breaking tasks into subtasks and optimizing only executed code segments, using the APPS+ dataset for training.
Large Language Models (LLMs) reinforcement learning (RL) compiler feedback code generation Curriculum of Code Completion Subtasks (CCCS) Fine-Grained Optimization (FGO) APPS+ dataset