AdvPrompter: Fast Adaptive Adversarial Prompting for LLMs
While recently Large Language Models (LLMs) have achieved remarkable
successes, they are vulnerable to certain jailbreaking attacks that lead to
generation of inappropriate or harmful content. Manual red-teaming requires
finding adversarial prompts that cause such jailbreaking, e.g. by appending a
suffix to a given instruction, which is inefficient and time-consuming. On the
other hand, automatic adversarial prompt generation often leads to semantically
meaningless attacks that can easily be detected by perplexity-based filters,
may require gradient information from the TargetLLM, or do not scale well due
to time-consuming discrete optimization processes over the token space. In this
paper, we present a novel method that uses another LLM, called the AdvPrompter,
to generate human-readable adversarial prompts in seconds, sim800times
faster than existing optimization-based approaches. We train the AdvPrompter
using a novel algorithm that does not require access to the gradients of the
TargetLLM. This process alternates between two steps: (1) generating
high-quality target adversarial suffixes by optimizing the AdvPrompter
predictions, and (2) low-rank fine-tuning of the AdvPrompter with the generated
adversarial suffixes. The trained AdvPrompter generates suffixes that veil the
input instruction without changing its meaning, such that the TargetLLM is
lured to give a harmful response. Experimental results on popular open source
TargetLLMs show state-of-the-art results on the AdvBench dataset, that also
transfer to closed-source black-box LLM APIs. Further, we demonstrate that by
fine-tuning on a synthetic dataset generated by AdvPrompter, LLMs can be made
more robust against jailbreaking attacks while maintaining performance, i.e.
high MMLU scores.
662efb293f9804ca5f91efbd
Anselm Paulus
662efb293f9804ca5f91efbe
Arman Zharmagambetov
claimed_verified
2024-04-29T11:46:52.541Z
64bed1fee10455688f2fd558
/avatars/1a9894c43955678c796473933bcd481f.svg
Arman Zharmagambetov
user
armanz
662efb293f9804ca5f91efbf
Chuan Guo
admin_assigned
2024-04-29T12:01:43.002Z
64c933cfdc59e43a6109b065
/avatars/b7f64fae8df8704c4911065759abfcd3.svg
Chuan Guo
user
chuanguo
662efb293f9804ca5f91efc0
Brandon Amos
admin_assigned
2024-04-29T12:01:49.558Z
63c765a550cc81901dad7b57
https://cdn-avatars.huggingface.co/v1/production/uploads/63c765a550cc81901dad7b57/i7QP4L7AtasyodWtx5Af0.jpeg
Brandon Amos
user
bamos
662efb293f9804ca5f91efc1
Yuandong Tian
admin_assigned
2024-04-29T12:01:55.534Z
6344cf73ee1504dbcd5bdfe7
/avatars/6dd2bf1f9c5679e5c8c85d62c9836aac.svg
Yuandong Tian
user
tydsh
AdvPrompter: Fast Adaptive Adversarial Prompting for LLMs
While recently Large Language Models (LLMs) have achieved remarkable
successes, they are vulnerable to certain jailbreaking attacks that lead to
generation of inappropriate or harmful content. Manual red-teaming requires
finding adversarial prompts that cause such jailbreaking, e.g. by appending a
suffix to a given instruction, which is inefficient and time-consuming. On the
other hand, automatic adversarial prompt generation often leads to semantically
meaningless attacks that can easily be detected by perplexity-based filters,
may require gradient information from the TargetLLM, or do not scale well due
to time-consuming discrete optimization processes over the token space. In this
paper, we present a novel method that uses another LLM, called the AdvPrompter,
to generate human-readable adversarial prompts in seconds, sim800times
faster than existing optimization-based approaches. We train the AdvPrompter
using a novel algorithm that does not require access to the gradients of the
TargetLLM. This process alternates between two steps: (1) generating
high-quality target adversarial suffixes by optimizing the AdvPrompter
predictions, and (2) low-rank fine-tuning of the AdvPrompter with the generated
adversarial suffixes. The trained AdvPrompter generates suffixes that veil the
input instruction without changing its meaning, such that the TargetLLM is
lured to give a harmful response. Experimental results on popular open source
TargetLLMs show state-of-the-art results on the AdvBench dataset, that also
transfer to closed-source black-box LLM APIs. Further, we demonstrate that by
fine-tuning on a synthetic dataset generated by AdvPrompter, LLMs can be made
more robust against jailbreaking attacks while maintaining performance, i.e.
high MMLU scores.
A novel method uses AdvPrompter, another LLM, to generate human-readable adversarial prompts efficiently, improving the state-of-the-art in generating unpredictable yet harmful outputs, and enhancing LLM robustness against jailbreaking attacks.
Large Language Models jailbreaking attacks adversarial prompts AdvPrompter perplexity-based filters discrete optimization low-rank fine-tuning AdvBench dataset MMLU scores synthetic dataset