Imperceptible Jailbreaking against Large Language Models
Jailbreaking attacks on the vision modality typically rely on imperceptible
adversarial perturbations, whereas attacks on the textual modality are
generally assumed to require visible modifications (e.g., non-semantic
suffixes). In this paper, we introduce imperceptible jailbreaks that exploit a
class of Unicode characters called variation selectors. By appending invisible
variation selectors to malicious questions, the jailbreak prompts appear
visually identical to original malicious questions on screen, while their
tokenization is "secretly" altered. We propose a chain-of-search pipeline to
generate such adversarial suffixes to induce harmful responses. Our experiments
show that our imperceptible jailbreaks achieve high attack success rates
against four aligned LLMs and generalize to prompt injection attacks, all
without producing any visible modifications in the written prompt. Our code is
available at https://github.com/sail-sg/imperceptible-jailbreaks.
68e4bd9ce4e093a7044e4e48
Kuofeng Gao
68e4bd9ce4e093a7044e4e49
Yiming Li
68e4bd9ce4e093a7044e4e4a
Chao Du
68e4bd9ce4e093a7044e4e4b
Xin Wang
68e4bd9ce4e093a7044e4e4c
Xingjun Ma
68e4bd9ce4e093a7044e4e4d
Shu-Tao Xia
68e4bd9ce4e093a7044e4e4e
Tianyu Pang
Imperceptible Jailbreaking against Large Language Models
Jailbreaking attacks on the vision modality typically rely on imperceptible
adversarial perturbations, whereas attacks on the textual modality are
generally assumed to require visible modifications (e.g., non-semantic
suffixes). In this paper, we introduce imperceptible jailbreaks that exploit a
class of Unicode characters called variation selectors. By appending invisible
variation selectors to malicious questions, the jailbreak prompts appear
visually identical to original malicious questions on screen, while their
tokenization is "secretly" altered. We propose a chain-of-search pipeline to
generate such adversarial suffixes to induce harmful responses. Our experiments
show that our imperceptible jailbreaks achieve high attack success rates
against four aligned LLMs and generalize to prompt injection attacks, all
without producing any visible modifications in the written prompt. Our code is
available at https://github.com/sail-sg/imperceptible-jailbreaks.
Imperceptible jailbreaks using Unicode variation selectors enable high attack success rates against aligned LLMs without visible prompt modifications.
adversarial perturbations Unicode characters variation selectors tokenization chain-of-search pipeline prompt injection attacks aligned LLMs