AIDE: AI-Driven Exploration in the Space of Code
Machine learning, the foundation of modern artificial intelligence, has
driven innovations that have fundamentally transformed the world. Yet, behind
advancements lies a complex and often tedious process requiring labor and
compute intensive iteration and experimentation. Engineers and scientists
developing machine learning models spend much of their time on trial-and-error
tasks instead of conceptualizing innovative solutions or research hypotheses.
To address this challenge, we introduce AI-Driven Exploration (AIDE), a machine
learning engineering agent powered by large language models (LLMs). AIDE frames
machine learning engineering as a code optimization problem, and formulates
trial-and-error as a tree search in the space of potential solutions. By
strategically reusing and refining promising solutions, AIDE effectively trades
computational resources for enhanced performance, achieving state-of-the-art
results on multiple machine learning engineering benchmarks, including our
Kaggle evaluations, OpenAI MLE-Bench and METRs RE-Bench.
67b6e0829b29983879ad2312
Zhengyao Jiang
admin_assigned
2025-02-20T17:38:24.557Z
630384837b50dd9d0a3328dc
/avatars/17097a93ef403592bc07c0ff6712faf3.svg
Zhengyao Jiang
user
ZhengyaoJiang
67b6e0829b29983879ad2313
Dominik Schmidt
admin_assigned
2025-02-20T17:38:52.544Z
63042a72eedc089484c89aee
/avatars/cbbe3000e6fa783c395b23c6b67da5ab.svg
Dominik Schmidt
user
dominikschmidt
67b6e0829b29983879ad2314
Dhruv Srikanth
admin_assigned
2025-02-20T17:38:58.578Z
62f3203506b6e6f54b4720f5
/avatars/64a395d3391e1025dbdd945c33ceee94.svg
Dhruv Srikanth
user
dSrikanth
67b6e0829b29983879ad2315
Dixing Xu
claimed_verified
2025-02-20T15:52:52.450Z
65f7927e7bc58032aa5bda58
https://cdn-avatars.huggingface.co/v1/production/uploads/65f7927e7bc58032aa5bda58/JxUSj-J7YBwtgj6rqQtjn.jpeg
Dex Dixing Xu
user
dexhunter
67b6e0829b29983879ad2316
Ian Kaplan
67b6e0829b29983879ad2317
Deniss Jacenko
claimed_verified
2025-03-25T08:21:46.526Z
63442e49b7485028097f005d
https://cdn-avatars.huggingface.co/v1/production/uploads/63442e49b7485028097f005d/bu8BhKCzp08GvrgMQ6c-T.jpeg
Deniss Jacenko
user
jacenko
67b6e0829b29983879ad2318
Yuxiang Wu
AIDE: AI-Driven Exploration in the Space of Code
Machine learning, the foundation of modern artificial intelligence, has
driven innovations that have fundamentally transformed the world. Yet, behind
advancements lies a complex and often tedious process requiring labor and
compute intensive iteration and experimentation. Engineers and scientists
developing machine learning models spend much of their time on trial-and-error
tasks instead of conceptualizing innovative solutions or research hypotheses.
To address this challenge, we introduce AI-Driven Exploration (AIDE), a machine
learning engineering agent powered by large language models (LLMs). AIDE frames
machine learning engineering as a code optimization problem, and formulates
trial-and-error as a tree search in the space of potential solutions. By
strategically reusing and refining promising solutions, AIDE effectively trades
computational resources for enhanced performance, achieving state-of-the-art
results on multiple machine learning engineering benchmarks, including our
Kaggle evaluations, OpenAI MLE-Bench and METRs RE-Bench.
large language models (LLMs) code optimization tree search