Paper2Code: Automating Code Generation from Scientific Papers in Machine
  Learning
Despite the rapid growth of machine learning research, corresponding code
implementations are often unavailable, making it slow and labor-intensive for
researchers to reproduce results and build upon prior work. In the meantime,
recent Large Language Models (LLMs) excel at understanding scientific documents
and generating high-quality code. Inspired by this, we introduce PaperCoder, a
multi-agent LLM framework that transforms machine learning papers into
functional code repositories. PaperCoder operates in three stages: planning,
where it constructs a high-level roadmap, designs the system architecture with
diagrams, identifies file dependencies, and generates configuration files;
analysis, which focuses on interpreting implementation-specific details; and
generation, where modular, dependency-aware code is produced. Moreover, each
phase is instantiated through a set of specialized agents designed to
collaborate effectively across the pipeline. We then evaluate PaperCoder on
generating code implementations from machine learning papers based on both
model-based and human evaluations, specifically from the original paper
authors, with author-released repositories as ground truth if available. Our
results demonstrate the effectiveness of PaperCoder in creating high-quality,
faithful implementations. Furthermore, it consistently shows strengths in the
recently released PaperBench benchmark, surpassing strong baselines by
substantial margins.
680aee7bcf67477f2c00ca53
Minju Seo
claimed_verified
2025-04-25T08:34:41.304Z
64f7bf0c7565a69eb693ad1f
/avatars/aba6910aa39a3437a7f0df3f5cd49e6d.svg
minju
user
iaminju
680aee7bcf67477f2c00ca54
Jinheon Baek
claimed_verified
2025-04-25T08:34:38.982Z
63036b6c5c70c21d0ea79d48
/avatars/a7eb03f5cbd4eaa09fe807bbed8bc0f7.svg
Jinheon Baek
user
jinheon
680aee7bcf67477f2c00ca55
Seongyun Lee
claimed_verified
2025-04-26T08:53:43.030Z
6550c4f27bbfce1878f5f280
/avatars/0ecedbcd8a55b2c4abd1da9e741a6652.svg
seongyun_lee
user
Seongyun
680aee7bcf67477f2c00ca56
Sung Ju Hwang
Paper2Code: Automating Code Generation from Scientific Papers in Machine
  Learning
Despite the rapid growth of machine learning research, corresponding code
implementations are often unavailable, making it slow and labor-intensive for
researchers to reproduce results and build upon prior work. In the meantime,
recent Large Language Models (LLMs) excel at understanding scientific documents
and generating high-quality code. Inspired by this, we introduce PaperCoder, a
multi-agent LLM framework that transforms machine learning papers into
functional code repositories. PaperCoder operates in three stages: planning,
where it constructs a high-level roadmap, designs the system architecture with
diagrams, identifies file dependencies, and generates configuration files;
analysis, which focuses on interpreting implementation-specific details; and
generation, where modular, dependency-aware code is produced. Moreover, each
phase is instantiated through a set of specialized agents designed to
collaborate effectively across the pipeline. We then evaluate PaperCoder on
generating code implementations from machine learning papers based on both
model-based and human evaluations, specifically from the original paper
authors, with author-released repositories as ground truth if available. Our
results demonstrate the effectiveness of PaperCoder in creating high-quality,
faithful implementations. Furthermore, it consistently shows strengths in the
recently released PaperBench benchmark, surpassing strong baselines by
substantial margins.
PaperCoder is a multi-agent LLM framework that converts machine learning papers into functional code repositories through planning, analysis, and generation stages.
Large Language Models LLM PaperCoder multi-agent framework system architecture configuration files modular dependency-aware PaperBench benchmark