MART: Improving LLM Safety with Multi-round Automatic Red-Teaming
Red-teaming is a common practice for mitigating unsafe behaviors in Large
Language Models (LLMs), which involves thoroughly assessing LLMs to identify
potential flaws and addressing them with responsible and accurate responses.
While effective, manual red-teaming is costly, and existing automatic
red-teaming typically discovers safety risks without addressing them. In this
paper, we propose a Multi-round Automatic Red-Teaming (MART) method, which
incorporates both automatic adversarial prompt writing and safe response
generation, significantly increasing red-teaming scalability and the safety of
the target LLM. Specifically, an adversarial LLM and a target LLM interplay
with each other in an iterative manner, where the adversarial LLM aims to
generate challenging prompts that elicit unsafe responses from the target LLM,
while the target LLM is fine-tuned with safety aligned data on these
adversarial prompts. In each round, the adversarial LLM crafts better attacks
on the updated target LLM, while the target LLM also improves itself through
safety fine-tuning. On adversarial prompt benchmarks, the violation rate of an
LLM with limited safety alignment reduces up to 84.7% after 4 rounds of MART,
achieving comparable performance to LLMs with extensive adversarial prompt
writing. Notably, model helpfulness on non-adversarial prompts remains stable
throughout iterations, indicating the target LLM maintains strong performance
on instruction following.
65542e46ddcd3e36f416516b
Suyu Ge
admin_assigned
2023-11-15T09:41:32.970Z
64f400fd0ff19714e24caaef
/avatars/a690ae5b30377b8179b45839447f7f93.svg
Suyu Ge
user
suuyu
65542e46ddcd3e36f416516c
Chunting Zhou
admin_assigned
2023-11-15T09:43:02.669Z
64f8d07dd493d8b0d2c04ecd
/avatars/7112232dcadcd2a7c12ec635d781b3ea.svg
Zhou
user
Chunting
65542e46ddcd3e36f416516d
Rui Hou
admin_assigned
2023-11-15T09:41:49.302Z
6407d236a7bc7c3865adcc3a
/avatars/8b75cc07fd049acde5363d268c5b88d4.svg
Rui Hou
user
rayhou
65542e46ddcd3e36f416516e
Madian Khabsa
65542e46ddcd3e36f416516f
Yi-Chia Wang
admin_assigned
2023-11-15T09:42:00.449Z
641b9775abfce26bcf7d0753
/avatars/778e2f1bb7fd5a80bfcf3cfb6863aa4c.svg
Yi-Chia Wang
user
yichiawang
65542e46ddcd3e36f4165170
Qifan Wang
admin_assigned
2023-11-15T09:42:11.404Z
64b65c84c991daa61404abd7
/avatars/3592aa8774ea4e2836099e0701b4b78d.svg
WangQifan
user
Bayees
65542e46ddcd3e36f4165171
Jiawei Han
65542e46ddcd3e36f4165172
Yuning Mao
admin_assigned
2023-11-15T09:42:46.743Z
62bf46a630a1888a761f8619
/avatars/de6c656066b1e88d37882ce68a0fd25c.svg
Yuning Mao
user
yuningm
MART: Improving LLM Safety with Multi-round Automatic Red-Teaming
Red-teaming is a common practice for mitigating unsafe behaviors in Large
Language Models (LLMs), which involves thoroughly assessing LLMs to identify
potential flaws and addressing them with responsible and accurate responses.
While effective, manual red-teaming is costly, and existing automatic
red-teaming typically discovers safety risks without addressing them. In this
paper, we propose a Multi-round Automatic Red-Teaming (MART) method, which
incorporates both automatic adversarial prompt writing and safe response
generation, significantly increasing red-teaming scalability and the safety of
the target LLM. Specifically, an adversarial LLM and a target LLM interplay
with each other in an iterative manner, where the adversarial LLM aims to
generate challenging prompts that elicit unsafe responses from the target LLM,
while the target LLM is fine-tuned with safety aligned data on these
adversarial prompts. In each round, the adversarial LLM crafts better attacks
on the updated target LLM, while the target LLM also improves itself through
safety fine-tuning. On adversarial prompt benchmarks, the violation rate of an
LLM with limited safety alignment reduces up to 84.7% after 4 rounds of MART,
achieving comparable performance to LLMs with extensive adversarial prompt
writing. Notably, model helpfulness on non-adversarial prompts remains stable
throughout iterations, indicating the target LLM maintains strong performance
on instruction following.
A multi-round automatic red-teaming method uses iterative adversarial prompts and safety fine-tuning to enhance the safety of large language models with minimal impact on their helpfulness.
Large Language Models (LLMs) automatic red-teaming adversarial prompt writing safe response generation adversarial LLM safety alignment adversarial prompt benchmarks