ACON: Optimizing Context Compression for Long-horizon LLM Agents
Large language models (LLMs) are increasingly deployed as agents in dynamic,
real-world environments, where success requires both reasoning and effective
tool use. A central challenge for agentic tasks is the growing context length,
as agents must accumulate long histories of actions and observations. This
expansion raises costs and reduces efficiency in long-horizon tasks, yet prior
work on context compression has mostly focused on single-step tasks or narrow
applications. We introduce Agent Context Optimization (ACON), a unified
framework that optimally compresses both environment observations and
interaction histories into concise yet informative condensations. ACON
leverages compression guideline optimization in natural language space: given
paired trajectories where full context succeeds but compressed context fails,
capable LLMs analyze the causes of failure, and the compression guideline is
updated accordingly. Furthermore, we propose distilling the optimized LLM
compressor into smaller models to reduce the overhead of the additional module.
Experiments on AppWorld, OfficeBench, and Multi-objective QA show that ACON
reduces memory usage by 26-54% (peak tokens) while largely preserving task
performance, preserves over 95% of accuracy when distilled into smaller
compressors, and enhances smaller LMs as long-horizon agents with up to 46%
performance improvement.
68de1bef6024653e8a3ed1b8
Minki Kang
68de1bef6024653e8a3ed1b9
Wei-Ning Chen
68de1bef6024653e8a3ed1ba
Dongge Han
68de1bef6024653e8a3ed1bb
Huseyin A. Inan
68de1bef6024653e8a3ed1bc
Lukas Wutschitz
68de1bef6024653e8a3ed1bd
Yanzhi Chen
68de1bef6024653e8a3ed1be
Robert Sim
68de1bef6024653e8a3ed1bf
Saravan Rajmohan
ACON: Optimizing Context Compression for Long-horizon LLM Agents
Large language models (LLMs) are increasingly deployed as agents in dynamic,
real-world environments, where success requires both reasoning and effective
tool use. A central challenge for agentic tasks is the growing context length,
as agents must accumulate long histories of actions and observations. This
expansion raises costs and reduces efficiency in long-horizon tasks, yet prior
work on context compression has mostly focused on single-step tasks or narrow
applications. We introduce Agent Context Optimization (ACON), a unified
framework that optimally compresses both environment observations and
interaction histories into concise yet informative condensations. ACON
leverages compression guideline optimization in natural language space: given
paired trajectories where full context succeeds but compressed context fails,
capable LLMs analyze the causes of failure, and the compression guideline is
updated accordingly. Furthermore, we propose distilling the optimized LLM
compressor into smaller models to reduce the overhead of the additional module.
Experiments on AppWorld, OfficeBench, and Multi-objective QA show that ACON
reduces memory usage by 26-54% (peak tokens) while largely preserving task
performance, preserves over 95% of accuracy when distilled into smaller
compressors, and enhances smaller LMs as long-horizon agents with up to 46%
performance improvement.
Agent Context Optimization (ACON) compresses context in large language models for efficient long-horizon tasks by analyzing failure cases and distilling the compressor into smaller models.
Agent Context Optimization ACON context compression environment observations interaction histories compression guideline optimization natural language space paired trajectories LLM compressor distillation long-horizon tasks memory usage task performance accuracy performance improvement