What Limits Agentic Systems Efficiency?
Large Language Models (LLMs), such as OpenAI-o1 and DeepSeek-R1, have
demonstrated strong reasoning capabilities. To further enhance LLM
capabilities, recent agentic systems, such as Deep Research, incorporate web
interactions into LLM reasoning to mitigate uncertainties and reduce potential
errors. However, existing research predominantly focuses on reasoning
performance, often neglecting the efficiency of agentic systems. In this work,
we present a comprehensive empirical study that identifies efficiency
bottlenecks in web-interactive agentic systems. We decompose end-to-end latency
into two primary components: LLM API latency and web environment latency. We
conduct a comprehensive empirical study across 15 models and 5 providers to
demonstrate high variability in API-based agentic systems. We observe that web
environment latency can contribute as much as 53.7% to the overall latency in a
web-based agentic system. To improve latency, we propose SpecCache, a caching
framework augmented with speculative execution that can reduce web environment
overhead. Extensive evaluations on two standard benchmarks show that our
approach improves the cache hit rate by up to 58x compared to a random caching
strategy, while reducing web environment overhead by up to 3.2x, without
degrading agentic system performance.
68f719db24c44893631119bc
Song Bian
claimed_verified
2025-10-23T15:00:48.966Z
620ec8059291e41cab585a3d
https://cdn-avatars.huggingface.co/v1/production/uploads/620ec8059291e41cab585a3d/iuVrjdeghqo-lz0lRfw7l.jpeg
Song
user
NaiveUser
68f719db24c44893631119bd
Minghao Yan
68f719db24c44893631119be
Anand Jayarajan
68f719db24c44893631119bf
Gennady Pekhimenko
68f719db24c44893631119c0
Shivaram Venkataraman
What Limits Agentic Systems Efficiency?
Large Language Models (LLMs), such as OpenAI-o1 and DeepSeek-R1, have
demonstrated strong reasoning capabilities. To further enhance LLM
capabilities, recent agentic systems, such as Deep Research, incorporate web
interactions into LLM reasoning to mitigate uncertainties and reduce potential
errors. However, existing research predominantly focuses on reasoning
performance, often neglecting the efficiency of agentic systems. In this work,
we present a comprehensive empirical study that identifies efficiency
bottlenecks in web-interactive agentic systems. We decompose end-to-end latency
into two primary components: LLM API latency and web environment latency. We
conduct a comprehensive empirical study across 15 models and 5 providers to
demonstrate high variability in API-based agentic systems. We observe that web
environment latency can contribute as much as 53.7% to the overall latency in a
web-based agentic system. To improve latency, we propose SpecCache, a caching
framework augmented with speculative execution that can reduce web environment
overhead. Extensive evaluations on two standard benchmarks show that our
approach improves the cache hit rate by up to 58x compared to a random caching
strategy, while reducing web environment overhead by up to 3.2x, without
degrading agentic system performance.
A caching framework with speculative execution reduces web environment latency in web-interactive agentic systems without degrading performance.
Large Language Models LLM agentic systems web interactions end-to-end latency LLM API latency web environment latency SpecCache caching framework speculative execution cache hit rate web environment overhead