LLM-based Optimization of Compound AI Systems: A Survey
In a compound AI system, components such as an LLM call, a retriever, a code
interpreter, or tools are interconnected. The system's behavior is primarily
driven by parameters such as instructions or tool definitions. Recent
advancements enable end-to-end optimization of these parameters using an LLM.
Notably, leveraging an LLM as an optimizer is particularly efficient because it
avoids gradient computation and can generate complex code and instructions.
This paper presents a survey of the principles and emerging trends in LLM-based
optimization of compound AI systems. It covers archetypes of compound AI
systems, approaches to LLM-based end-to-end optimization, and insights into
future directions and broader impacts. Importantly, this survey uses concepts
from program analysis to provide a unified view of how an LLM optimizer is
prompted to optimize a compound AI system. The exhaustive list of paper is
provided at
https://github.com/linyuhongg/LLM-based-Optimization-of-Compound-AI-Systems.
6719941a2d24f0c6bb1cf9db
Matthieu Lin
6719941a2d24f0c6bb1cf9dc
Jenny Sheng
6719941a2d24f0c6bb1cf9dd
Andrew Zhao
claimed_verified
2025-04-22T10:17:25.420Z
630482fbce6b12280b18971d
/avatars/b07f31fd970d736bdf574d56da7a5634.svg
Andrew Zhao
user
andrewzh
6719941a2d24f0c6bb1cf9de
Shenzhi Wang
claimed_verified
2024-10-24T07:15:34.990Z
6486dde1f74857df3f1a5828
https://cdn-avatars.huggingface.co/v1/production/uploads/6486dde1f74857df3f1a5828/FgE80CpalBO5qqArdfwxA.jpeg
Shenzhi Wang
user
shenzhi-wang
6719941a2d24f0c6bb1cf9df
Yang Yue
claimed_verified
2024-11-05T07:58:48.839Z
649d475111592b1a765ac1a3
https://cdn-avatars.huggingface.co/v1/production/uploads/649d475111592b1a765ac1a3/rjORJjErJq-mthghan08U.jpeg
Yang Yue
user
Yang130
6719941a2d24f0c6bb1cf9e0
Yiran Wu
6719941a2d24f0c6bb1cf9e1
Huan Liu
6719941a2d24f0c6bb1cf9e2
Jun Liu
6719941a2d24f0c6bb1cf9e3
Gao Huang
6719941a2d24f0c6bb1cf9e4
Yong-Jin Liu
LLM-based Optimization of Compound AI Systems: A Survey
In a compound AI system, components such as an LLM call, a retriever, a code
interpreter, or tools are interconnected. The system's behavior is primarily
driven by parameters such as instructions or tool definitions. Recent
advancements enable end-to-end optimization of these parameters using an LLM.
Notably, leveraging an LLM as an optimizer is particularly efficient because it
avoids gradient computation and can generate complex code and instructions.
This paper presents a survey of the principles and emerging trends in LLM-based
optimization of compound AI systems. It covers archetypes of compound AI
systems, approaches to LLM-based end-to-end optimization, and insights into
future directions and broader impacts. Importantly, this survey uses concepts
from program analysis to provide a unified view of how an LLM optimizer is
prompted to optimize a compound AI system. The exhaustive list of paper is
provided at
https://github.com/linyuhongg/LLM-based-Optimization-of-Compound-AI-Systems.
A survey of principles and trends in using large language models (LLMs) for end-to-end optimization of compound AI systems, focusing on LLM-driven approaches and program analysis concepts.
LLM compound AI systems end-to-end optimization LLM-based optimization program analysis