PanGu-Coder2: Boosting Large Language Models for Code with Ranking
  Feedback
Large Language Models for Code (Code LLM) are flourishing. New and powerful
models are released on a weekly basis, demonstrating remarkable performance on
the code generation task. Various approaches have been proposed to boost the
code generation performance of pre-trained Code LLMs, such as supervised
fine-tuning, instruction tuning, reinforcement learning, etc. In this paper, we
propose a novel RRTF (Rank Responses to align Test&Teacher Feedback) framework,
which can effectively and efficiently boost pre-trained large language models
for code generation. Under this framework, we present PanGu-Coder2, which
achieves 62.20% pass@1 on the OpenAI HumanEval benchmark. Furthermore, through
an extensive evaluation on CoderEval and LeetCode benchmarks, we show that
PanGu-Coder2 consistently outperforms all previous Code LLMs.
64c316dbf33ee56c8067528d
Bo Shen
64c316dbf33ee56c8067528e
Jiaxin Zhang
claimed_verified
2023-07-29T17:45:55.917Z
62610bf040e04009e810357c
/avatars/16056719d57d4fcf7c3d21135e65f1a2.svg
Jiaxin Zhang
user
BerryBiscuits
64c316dbf33ee56c8067528f
Taihong Chen
64c316dbf33ee56c80675290
Daoguang Zan
claimed_verified
2023-09-04T07:32:24.288Z
61527edf8b55dbdae72874fa
https://cdn-avatars.huggingface.co/v1/production/uploads/61527edf8b55dbdae72874fa/ZGWSBf_KSrDof6WyMoDMU.jpeg
Daoguang Zan
user
Daoguang
64c316dbf33ee56c80675291
Bing Geng
claimed_verified
2023-07-29T17:45:59.413Z
646ec43b8d316fde87b3871a
/avatars/195cc396e160bb65ef051a2e5b6a0ebc.svg
Geng Bing
user
edwingeng
64c316dbf33ee56c80675292
An Fu
64c316dbf33ee56c80675293
Muhan Zeng
64c316dbf33ee56c80675294
Ailun Yu
64c316dbf33ee56c80675295
Jichuan Ji
64c316dbf33ee56c80675296
Jingyang Zhao
64c316dbf33ee56c80675297
Yuenan Guo
64c316dbf33ee56c80675298
Qianxiang Wang
PanGu-Coder2: Boosting Large Language Models for Code with Ranking
  Feedback
Large Language Models for Code (Code LLM) are flourishing. New and powerful
models are released on a weekly basis, demonstrating remarkable performance on
the code generation task. Various approaches have been proposed to boost the
code generation performance of pre-trained Code LLMs, such as supervised
fine-tuning, instruction tuning, reinforcement learning, etc. In this paper, we
propose a novel RRTF (Rank Responses to align Test&Teacher Feedback) framework,
which can effectively and efficiently boost pre-trained large language models
for code generation. Under this framework, we present PanGu-Coder2, which
achieves 62.20% pass@1 on the OpenAI HumanEval benchmark. Furthermore, through
an extensive evaluation on CoderEval and LeetCode benchmarks, we show that
PanGu-Coder2 consistently outperforms all previous Code LLMs.
PanGu-Coder2, a model fine-tuned with RRTF framework, demonstrates superior code generation performance, outperforming existing Code LLMs on multiple benchmarks.
RRTF pre-trained large language models code generation OpenAI HumanEval CoderEval LeetCode benchmarks