Compound AI Systems Optimization: A Survey of Methods, Challenges, and
  Future Directions
Recent advancements in large language models (LLMs) and AI systems have led
to a paradigm shift in the design and optimization of complex AI workflows. By
integrating multiple components, compound AI systems have become increasingly
adept at performing sophisticated tasks. However, as these systems grow in
complexity, new challenges arise in optimizing not only individual components
but also their interactions. While traditional optimization methods such as
supervised fine-tuning (SFT) and reinforcement learning (RL) remain
foundational, the rise of natural language feedback introduces promising new
approaches, especially for optimizing non-differentiable systems. This paper
provides a systematic review of recent progress in optimizing compound AI
systems, encompassing both numerical and language-based techniques. We
formalize the notion of compound AI system optimization, classify existing
methods along several key dimensions, and highlight open research challenges
and future directions in this rapidly evolving field. A list of surveyed papers
is publicly available at https://github.com/MiuLab/AISysOpt-Survey.
684ae1dddbd21a9cc27b0edc
Yu-Ang Lee
claimed_verified
2025-06-13T14:59:40.713Z
6615752da15c52fa7ab3e2f7
/avatars/37e72cfb829a42630d229080ad8d60f3.svg
Lee
user
Speeeed
684ae1dddbd21a9cc27b0edd
Guan-Ting Yi
684ae1dddbd21a9cc27b0ede
Mei-Yi Liu
684ae1dddbd21a9cc27b0edf
Jui-Chao Lu
684ae1dddbd21a9cc27b0ee0
Guan-Bo Yang
684ae1dddbd21a9cc27b0ee1
Yun-Nung Chen
Compound AI Systems Optimization: A Survey of Methods, Challenges, and
  Future Directions
Recent advancements in large language models (LLMs) and AI systems have led
to a paradigm shift in the design and optimization of complex AI workflows. By
integrating multiple components, compound AI systems have become increasingly
adept at performing sophisticated tasks. However, as these systems grow in
complexity, new challenges arise in optimizing not only individual components
but also their interactions. While traditional optimization methods such as
supervised fine-tuning (SFT) and reinforcement learning (RL) remain
foundational, the rise of natural language feedback introduces promising new
approaches, especially for optimizing non-differentiable systems. This paper
provides a systematic review of recent progress in optimizing compound AI
systems, encompassing both numerical and language-based techniques. We
formalize the notion of compound AI system optimization, classify existing
methods along several key dimensions, and highlight open research challenges
and future directions in this rapidly evolving field. A list of surveyed papers
is publicly available at https://github.com/MiuLab/AISysOpt-Survey.
Recent advancements in optimizing compound AI systems highlight challenges in integrating various components, with an emphasis on natural language feedback methods for non-differentiable systems.
large language models AI systems compound AI systems supervised fine-tuning reinforcement learning natural language feedback non-differentiable systems