FocusAgent: Simple Yet Effective Ways of Trimming the Large Context of
  Web Agents
Web agents powered by large language models (LLMs) must process lengthy web
page observations to complete user goals; these pages often exceed tens of
thousands of tokens. This saturates context limits and increases computational
cost processing; moreover, processing full pages exposes agents to security
risks such as prompt injection. Existing pruning strategies either discard
relevant content or retain irrelevant context, leading to suboptimal action
prediction. We introduce FocusAgent, a simple yet effective approach that
leverages a lightweight LLM retriever to extract the most relevant lines from
accessibility tree (AxTree) observations, guided by task goals. By pruning
noisy and irrelevant content, FocusAgent enables efficient reasoning while
reducing vulnerability to injection attacks. Experiments on WorkArena and
WebArena benchmarks show that FocusAgent matches the performance of strong
baselines, while reducing observation size by over 50%. Furthermore, a variant
of FocusAgent significantly reduces the success rate of prompt-injection
attacks, including banner and pop-up attacks, while maintaining task success
performance in attack-free settings. Our results highlight that targeted
LLM-based retrieval is a practical and robust strategy for building web agents
that are efficient, effective, and secure.
68e323b473e20ab577841fcf
Imene Kerboua
claimed_verified
2025-10-07T12:28:31.990Z
62610f8040e04009e81047e9
https://cdn-avatars.huggingface.co/v1/production/uploads/62610f8040e04009e81047e9/iqGTYB7OMmS0jkFEB7cEF.jpeg
Imene Kerboua
user
imenelydiaker
68e323b473e20ab577841fd0
Sahar Omidi Shayegan
68e323b473e20ab577841fd1
Megh Thakkar
68e323b473e20ab577841fd2
Xing Han Lù
claimed_verified
2025-10-15T03:14:32.853Z
5fa9ff3ea13e063b8b2b60cb
https://cdn-avatars.huggingface.co/v1/production/uploads/1633380224986-5fa9ff3ea13e063b8b2b60cb.jpeg
Xing Han Lù
user
xhluca
68e323b473e20ab577841fd3
Léo Boisvert
68e323b473e20ab577841fd4
Massimo Caccia
68e323b473e20ab577841fd5
Jérémy Espinas
68e323b473e20ab577841fd6
Alexandre Aussem
68e323b473e20ab577841fd7
Véronique Eglin
68e323b473e20ab577841fd8
Alexandre Lacoste
FocusAgent: Simple Yet Effective Ways of Trimming the Large Context of
  Web Agents
Web agents powered by large language models (LLMs) must process lengthy web
page observations to complete user goals; these pages often exceed tens of
thousands of tokens. This saturates context limits and increases computational
cost processing; moreover, processing full pages exposes agents to security
risks such as prompt injection. Existing pruning strategies either discard
relevant content or retain irrelevant context, leading to suboptimal action
prediction. We introduce FocusAgent, a simple yet effective approach that
leverages a lightweight LLM retriever to extract the most relevant lines from
accessibility tree (AxTree) observations, guided by task goals. By pruning
noisy and irrelevant content, FocusAgent enables efficient reasoning while
reducing vulnerability to injection attacks. Experiments on WorkArena and
WebArena benchmarks show that FocusAgent matches the performance of strong
baselines, while reducing observation size by over 50%. Furthermore, a variant
of FocusAgent significantly reduces the success rate of prompt-injection
attacks, including banner and pop-up attacks, while maintaining task success
performance in attack-free settings. Our results highlight that targeted
LLM-based retrieval is a practical and robust strategy for building web agents
that are efficient, effective, and secure.
FocusAgent uses a lightweight LLM retriever to extract relevant content from web page observations, improving efficiency and security in web agents.
large language models LLM accessibility tree AxTree prompt injection WorkArena WebArena targeted LLM-based retrieval