CodexGraph: Bridging Large Language Models and Code Repositories via
  Code Graph Databases
Large Language Models (LLMs) excel in stand-alone code tasks like HumanEval
and MBPP, but struggle with handling entire code repositories. This challenge
has prompted research on enhancing LLM-codebase interaction at a repository
scale. Current solutions rely on similarity-based retrieval or manual tools and
APIs, each with notable drawbacks. Similarity-based retrieval often has low
recall in complex tasks, while manual tools and APIs are typically
task-specific and require expert knowledge, reducing their generalizability
across diverse code tasks and real-world applications. To mitigate these
limitations, we introduce \framework, a system that integrates LLM agents with
graph database interfaces extracted from code repositories. By leveraging the
structural properties of graph databases and the flexibility of the graph query
language, \framework enables the LLM agent to construct and execute queries,
allowing for precise, code structure-aware context retrieval and code
navigation. We assess \framework using three benchmarks: CrossCodeEval,
SWE-bench, and EvoCodeBench. Additionally, we develop five real-world coding
applications. With a unified graph database schema, \framework demonstrates
competitive performance and potential in both academic and real-world
environments, showcasing its versatility and efficacy in software engineering.
Our application demo:
https://github.com/modelscope/modelscope-agent/tree/master/apps/codexgraph_agent.
66b42c9b1c879ddf19205604
Xiangyan Liu
admin_assigned
2024-08-08T10:50:54.107Z
6486b09e8315b19342f0bf5e
/avatars/bc5f22f231c884146d373fe1042d81bd.svg
Xiangyan Liu
user
xyliu6
66b42c9b1c879ddf19205605
Bo Lan
66b42c9b1c879ddf19205606
Zhiyuan Hu
admin_assigned
2024-08-08T10:51:45.023Z
65e9945f6bcbcae600b7e64f
/avatars/8aa986a6c0e35c55d5d1461d1dc11ac3.svg
Zhiyuan Hu
user
zhiyhu
66b42c9b1c879ddf19205607
Yang Liu
66b42c9b1c879ddf19205608
Zhicheng Zhang
66b42c9b1c879ddf19205609
Wenmeng Zhou
admin_assigned
2024-08-08T10:52:24.365Z
623c6253389748c9f72ca287
https://cdn-avatars.huggingface.co/v1/production/uploads/1654828369523-623c6253389748c9f72ca287.jpeg
wenmeng zhou
user
wenmengzhou
66b42c9b1c879ddf1920560a
Fei Wang
66b42c9b1c879ddf1920560b
Michael Shieh
admin_assigned
2024-08-08T10:53:41.871Z
64d48e9ef2a3941669810f23
/avatars/495e6785abc44d1facfcf385edd6a222.svg
Michael Shieh
user
Michael2008S
CodexGraph: Bridging Large Language Models and Code Repositories via
  Code Graph Databases
Large Language Models (LLMs) excel in stand-alone code tasks like HumanEval
and MBPP, but struggle with handling entire code repositories. This challenge
has prompted research on enhancing LLM-codebase interaction at a repository
scale. Current solutions rely on similarity-based retrieval or manual tools and
APIs, each with notable drawbacks. Similarity-based retrieval often has low
recall in complex tasks, while manual tools and APIs are typically
task-specific and require expert knowledge, reducing their generalizability
across diverse code tasks and real-world applications. To mitigate these
limitations, we introduce \framework, a system that integrates LLM agents with
graph database interfaces extracted from code repositories. By leveraging the
structural properties of graph databases and the flexibility of the graph query
language, \framework enables the LLM agent to construct and execute queries,
allowing for precise, code structure-aware context retrieval and code
navigation. We assess \framework using three benchmarks: CrossCodeEval,
SWE-bench, and EvoCodeBench. Additionally, we develop five real-world coding
applications. With a unified graph database schema, \framework demonstrates
competitive performance and potential in both academic and real-world
environments, showcasing its versatility and efficacy in software engineering.
Our application demo:
https://github.com/modelscope/modelscope-agent/tree/master/apps/codexgraph_agent.
A system integrating LLMs with graph database interfaces for improved codebase interaction demonstrates competitive performance in real-world coding applications.
Large Language Models LLMs HumanEval MBPP similarity-based retrieval graph database interfaces graph query language CrossCodeEval SWE-bench EvoCodeBench code structure-aware context retrieval code navigation