Testing Language Model Agents Safely in the Wild
A prerequisite for safe autonomy-in-the-wild is safe testing-in-the-wild. Yet
real-world autonomous tests face several unique safety challenges, both due to
the possibility of causing harm during a test, as well as the risk of
encountering new unsafe agent behavior through interactions with real-world and
potentially malicious actors. We propose a framework for conducting safe
autonomous agent tests on the open internet: agent actions are audited by a
context-sensitive monitor that enforces a stringent safety boundary to stop an
unsafe test, with suspect behavior ranked and logged to be examined by humans.
We a design a basic safety monitor that is flexible enough to monitor existing
LLM agents, and, using an adversarial simulated agent, we measure its ability
to identify and stop unsafe situations. Then we apply the safety monitor on a
battery of real-world tests of AutoGPT, and we identify several limitations and
challenges that will face the creation of safe in-the-wild tests as autonomous
agents grow more capable.
655ac2907f2466433989e3ce
Silen Naihin
admin_assigned
2023-11-20T13:48:17.014Z
63754055fd59a439194ccd8c
/avatars/f929cc5f7eeabd343a89d24d0a83f6b2.svg
Silen Naihin
user
Silen
655ac2907f2466433989e3cf
David Atkinson
claimed_verified
2024-04-12T14:48:54.148Z
64769e3c3d8d1cb791909f09
/avatars/c09f6f1df52a710db6a65e37a4545458.svg
David Atkinson
user
datkinson
655ac2907f2466433989e3d0
Marc Green
claimed_verified
2024-04-16T00:01:30.136Z
637fdccc3b36a53c22ec7eee
/avatars/261e07f6a312de46383df78b9af11b74.svg
Marc Green
user
marcgreen
655ac2907f2466433989e3d1
Merwane Hamadi
admin_assigned
2023-11-20T13:49:03.778Z
64405061757aa3c2ad863726
/avatars/eb2bf2f5d6d5b3ed07deeb0c7e29b6fd.svg
Merwane Hamadi
user
merwanehamadi
655ac2907f2466433989e3d2
Craig Swift
admin_assigned
2023-11-20T13:49:14.708Z
6467fee96192d391422669d5
/avatars/595133486c0d73eb21138cbd0a49ae48.svg
Craig Swift
user
SwiftyOS
655ac2907f2466433989e3d3
Douglas Schonholtz
admin_assigned
2023-11-20T13:49:21.876Z
6372a17cacef705233c88a50
/avatars/a428dd78ccf0d962c37e66bf343ec090.svg
Douglas Schonholtz
user
dschon
655ac2907f2466433989e3d4
Adam Tauman Kalai
admin_assigned
2023-11-20T13:49:27.932Z
64e3b46d618cd90997e9d342
/avatars/692aa11b62ac880833a8a183018bec70.svg
Adam Tauman Kalai
user
adamology
655ac2907f2466433989e3d5
David Bau
admin_assigned
2023-11-20T13:50:02.882Z
6214d6c01e35c843d42d1f77
/avatars/ac208cd180b4f3ed1ec367e581facfcf.svg
David Bau
user
davidbau
Testing Language Model Agents Safely in the Wild
A prerequisite for safe autonomy-in-the-wild is safe testing-in-the-wild. Yet
real-world autonomous tests face several unique safety challenges, both due to
the possibility of causing harm during a test, as well as the risk of
encountering new unsafe agent behavior through interactions with real-world and
potentially malicious actors. We propose a framework for conducting safe
autonomous agent tests on the open internet: agent actions are audited by a
context-sensitive monitor that enforces a stringent safety boundary to stop an
unsafe test, with suspect behavior ranked and logged to be examined by humans.
We a design a basic safety monitor that is flexible enough to monitor existing
LLM agents, and, using an adversarial simulated agent, we measure its ability
to identify and stop unsafe situations. Then we apply the safety monitor on a
battery of real-world tests of AutoGPT, and we identify several limitations and
challenges that will face the creation of safe in-the-wild tests as autonomous
agents grow more capable.
A safety framework with a context-sensitive monitor is proposed to conduct safe testing of autonomous agents in the real world, identifying and stopping unsafe behaviors through adversarial simulations and real-world tests.
autonomous agents safety monitoring adversarial simulated agent AutoGPT