CODESIM: Multi-Agent Code Generation and Problem Solving through
  Simulation-Driven Planning and Debugging
Large Language Models (LLMs) have made significant strides in code generation
and problem solving. Current approaches employ external tool-based iterative
debuggers that use compiler or other tool-based runtime feedback to refine
coarse programs generated by various methods. However, the effectiveness of
these approaches heavily relies on the quality of the initial code generation,
which remains an open challenge. In this paper, we introduce CodeSim, a novel
multi-agent code generation framework that comprehensively addresses the stages
of program synthesis-planning, coding, and debugging-through a human-like
perception approach. As human verifies their understanding of any algorithms
through visual simulation, CodeSim uniquely features a method of plan
verification and internal debugging through the step-by-step simulation of
input/output. Extensive experiments across seven challenging competitive
problem-solving and program synthesis benchmarks demonstrate CodeSim's
remarkable code generation capabilities. Our framework achieves new
state-of-the-art (pass@1) results-(HumanEval 95.1%, MBPP 90.7%, APPS 22%, and
CodeContests 29.1%). Furthermore, our method shows potential for even greater
enhancement when cascaded with external debuggers. To facilitate further
research and development in this area, we have open-sourced our framework in
this link (https://kagnlp.github.io/codesim.github.io/).
67ab56dc0bc5f6a94eb49892
Md. Ashraful Islam
claimed_verified
2025-02-11T14:25:21.564Z
63fe51dcc0ec83fda436d558
https://cdn-avatars.huggingface.co/v1/production/uploads/63fe51dcc0ec83fda436d558/22wrFA08OxRLIsRVxPts0.jpeg
Md. Ashraful Islam
user
ashraful
67ab56dc0bc5f6a94eb49893
Mohammed Eunus Ali
67ab56dc0bc5f6a94eb49894
Md Rizwan Parvez
CODESIM: Multi-Agent Code Generation and Problem Solving through
  Simulation-Driven Planning and Debugging
Large Language Models (LLMs) have made significant strides in code generation
and problem solving. Current approaches employ external tool-based iterative
debuggers that use compiler or other tool-based runtime feedback to refine
coarse programs generated by various methods. However, the effectiveness of
these approaches heavily relies on the quality of the initial code generation,
which remains an open challenge. In this paper, we introduce CodeSim, a novel
multi-agent code generation framework that comprehensively addresses the stages
of program synthesis-planning, coding, and debugging-through a human-like
perception approach. As human verifies their understanding of any algorithms
through visual simulation, CodeSim uniquely features a method of plan
verification and internal debugging through the step-by-step simulation of
input/output. Extensive experiments across seven challenging competitive
problem-solving and program synthesis benchmarks demonstrate CodeSim's
remarkable code generation capabilities. Our framework achieves new
state-of-the-art (pass@1) results-(HumanEval 95.1%, MBPP 90.7%, APPS 22%, and
CodeContests 29.1%). Furthermore, our method shows potential for even greater
enhancement when cascaded with external debuggers. To facilitate further
research and development in this area, we have open-sourced our framework in
this link (https://kagnlp.github.io/codesim.github.io/).
CodeSim, a multi-agent framework, improves code generation and debugging through human-like perception and step-by-step simulation, achieving state-of-the-art results across several benchmarks.
multi-agent framework program synthesis plan verification internal debugging step-by-step simulation HumanEval MBPP APPS CodeContests