Progent: Programmable Privilege Control for LLM Agents
LLM agents are an emerging form of AI systems where large language models
(LLMs) serve as the central component, utilizing a diverse set of tools to
complete user-assigned tasks. Despite their great potential, LLM agents pose
significant security risks. When interacting with the external world, they may
encounter malicious commands from attackers, leading to the execution of
dangerous actions. A promising way to address this is by enforcing the
principle of least privilege: allowing only essential actions for task
completion while blocking unnecessary ones. However, achieving this is
challenging, as it requires covering diverse agent scenarios while preserving
both security and utility.
  We introduce Progent, the first privilege control mechanism for LLM agents.
At its core is a domain-specific language for flexibly expressing privilege
control policies applied during agent execution. These policies provide
fine-grained constraints over tool calls, deciding when tool calls are
permissible and specifying fallbacks if they are not. This enables agent
developers and users to craft suitable policies for their specific use cases
and enforce them deterministically to guarantee security. Thanks to its modular
design, integrating Progent does not alter agent internals and requires only
minimal changes to agent implementation, enhancing its practicality and
potential for widespread adoption. To automate policy writing, we leverage LLMs
to generate policies based on user queries, which are then updated dynamically
for improved security and utility. Our extensive evaluation shows that it
enables strong security while preserving high utility across three distinct
scenarios or benchmarks: AgentDojo, ASB, and AgentPoison. Furthermore, we
perform an in-depth analysis, showcasing the effectiveness of its core
components and the resilience of its automated policy generation against
adaptive attacks.
6800a4f9f16f9f820ed748af
Tianneng Shi
claimed_verified
2025-04-22T10:13:56.123Z
64f27f74f1b6c235aed4b904
/avatars/1129af4fcf566b87a5ff81375376bf3a.svg
stneng
user
stneng
6800a4f9f16f9f820ed748b0
Jingxuan He
6800a4f9f16f9f820ed748b1
Zhun Wang
6800a4f9f16f9f820ed748b2
Linyu Wu
6800a4f9f16f9f820ed748b3
Hongwei Li
6800a4f9f16f9f820ed748b4
Wenbo Guo
6800a4f9f16f9f820ed748b5
Dawn Song
Progent: Programmable Privilege Control for LLM Agents
LLM agents are an emerging form of AI systems where large language models
(LLMs) serve as the central component, utilizing a diverse set of tools to
complete user-assigned tasks. Despite their great potential, LLM agents pose
significant security risks. When interacting with the external world, they may
encounter malicious commands from attackers, leading to the execution of
dangerous actions. A promising way to address this is by enforcing the
principle of least privilege: allowing only essential actions for task
completion while blocking unnecessary ones. However, achieving this is
challenging, as it requires covering diverse agent scenarios while preserving
both security and utility.
  We introduce Progent, the first privilege control mechanism for LLM agents.
At its core is a domain-specific language for flexibly expressing privilege
control policies applied during agent execution. These policies provide
fine-grained constraints over tool calls, deciding when tool calls are
permissible and specifying fallbacks if they are not. This enables agent
developers and users to craft suitable policies for their specific use cases
and enforce them deterministically to guarantee security. Thanks to its modular
design, integrating Progent does not alter agent internals and requires only
minimal changes to agent implementation, enhancing its practicality and
potential for widespread adoption. To automate policy writing, we leverage LLMs
to generate policies based on user queries, which are then updated dynamically
for improved security and utility. Our extensive evaluation shows that it
enables strong security while preserving high utility across three distinct
scenarios or benchmarks: AgentDojo, ASB, and AgentPoison. Furthermore, we
perform an in-depth analysis, showcasing the effectiveness of its core
components and the resilience of its automated policy generation against
adaptive attacks.
Progent is a privilege control mechanism for LLM agents that enforces fine-grained tool call policies using a domain-specific language, ensuring security and utility across various scenarios.
LLM agents privilege control mechanism domain-specific language tool calls policy writing automated policy generation security utility AgentDojo ASB AgentPoison