From Code to Correctness: Closing the Last Mile of Code Generation with
  Hierarchical Debugging
While large language models have made significant strides in code generation,
the pass rate of the generated code is bottlenecked on subtle errors, often
requiring human intervention to pass tests, especially for complex problems.
Existing LLM-based debugging systems treat generated programs as monolithic
units, failing to address bugs at multiple levels of granularity, from
low-level syntax errors to high-level algorithmic flaws. In this paper, we
introduce Multi-Granularity Debugger (MGDebugger), a hierarchical code debugger
by isolating, identifying, and resolving bugs at various levels of granularity.
MGDebugger decomposes problematic code into a hierarchical tree structure of
subfunctions, with each level representing a particular granularity of error.
During debugging, it analyzes each subfunction and iteratively resolves bugs in
a bottom-up manner. To effectively test each subfunction, we propose an
LLM-simulated Python executor, which traces code execution and tracks important
variable states to pinpoint errors accurately. Extensive experiments
demonstrate that MGDebugger outperforms existing debugging systems, achieving
an 18.9% improvement in accuracy over seed generations in HumanEval and a 97.6%
repair success rate in HumanEvalFix. Furthermore, MGDebugger effectively fixes
bugs across different categories and difficulty levels, demonstrating its
robustness and effectiveness.
66fe140c7d722f0879868725
Yuling Shi
claimed_verified
2024-10-03T08:31:20.232Z
645b0c3ec35da9c7afd95421
https://cdn-avatars.huggingface.co/v1/production/uploads/645b0c3ec35da9c7afd95421/vYBrCDagHsXAo6J2p-uG0.jpeg
Yuling
user
YerbaPage
66fe140c7d722f0879868726
Songsong Wang
66fe140c7d722f0879868727
Chengcheng Wan
66fe140c7d722f0879868728
Xiaodong Gu
claimed_verified
2024-10-07T07:28:40.543Z
6624ede5478dada13051af60
/avatars/1b064f7e3c417f1b603909ff3dbf2430.svg
xiaodong gu
user
guxiaodong1987
From Code to Correctness: Closing the Last Mile of Code Generation with
  Hierarchical Debugging
While large language models have made significant strides in code generation,
the pass rate of the generated code is bottlenecked on subtle errors, often
requiring human intervention to pass tests, especially for complex problems.
Existing LLM-based debugging systems treat generated programs as monolithic
units, failing to address bugs at multiple levels of granularity, from
low-level syntax errors to high-level algorithmic flaws. In this paper, we
introduce Multi-Granularity Debugger (MGDebugger), a hierarchical code debugger
by isolating, identifying, and resolving bugs at various levels of granularity.
MGDebugger decomposes problematic code into a hierarchical tree structure of
subfunctions, with each level representing a particular granularity of error.
During debugging, it analyzes each subfunction and iteratively resolves bugs in
a bottom-up manner. To effectively test each subfunction, we propose an
LLM-simulated Python executor, which traces code execution and tracks important
variable states to pinpoint errors accurately. Extensive experiments
demonstrate that MGDebugger outperforms existing debugging systems, achieving
an 18.9% improvement in accuracy over seed generations in HumanEval and a 97.6%
repair success rate in HumanEvalFix. Furthermore, MGDebugger effectively fixes
bugs across different categories and difficulty levels, demonstrating its
robustness and effectiveness.
MGDebugger hierarchical debugger resolves code bugs at multiple granularities, improving debugging accuracy and success rates over existing systems.
large language models code generation monolithic units syntax errors algorithmic flaws hierarchical code debugger subfunctions LLM-simulated Python executor variable states HumanEval HumanEvalFix