QGuard:Question-based Zero-shot Guard for Multi-modal LLM Safety
The recent advancements in Large Language Models(LLMs) have had a significant
impact on a wide range of fields, from general domains to specialized areas.
However, these advancements have also significantly increased the potential for
malicious users to exploit harmful and jailbreak prompts for malicious attacks.
Although there have been many efforts to prevent harmful prompts and jailbreak
prompts, protecting LLMs from such malicious attacks remains an important and
challenging task. In this paper, we propose QGuard, a simple yet effective
safety guard method, that utilizes question prompting to block harmful prompts
in a zero-shot manner. Our method can defend LLMs not only from text-based
harmful prompts but also from multi-modal harmful prompt attacks. Moreover, by
diversifying and modifying guard questions, our approach remains robust against
the latest harmful prompts without fine-tuning. Experimental results show that
our model performs competitively on both text-only and multi-modal harmful
datasets. Additionally, by providing an analysis of question prompting, we
enable a white-box analysis of user inputs. We believe our method provides
valuable insights for real-world LLM services in mitigating security risks
associated with harmful prompts.
6850e32c5e07650ecce89112
Taegyeong Lee
6850e32c5e07650ecce89113
Jeonghwa Yoo
6850e32c5e07650ecce89114
Hyoungseo Cho
6850e32c5e07650ecce89115
Soo Yong Kim
6850e32c5e07650ecce89116
Yunho Maeng
QGuard:Question-based Zero-shot Guard for Multi-modal LLM Safety
The recent advancements in Large Language Models(LLMs) have had a significant
impact on a wide range of fields, from general domains to specialized areas.
However, these advancements have also significantly increased the potential for
malicious users to exploit harmful and jailbreak prompts for malicious attacks.
Although there have been many efforts to prevent harmful prompts and jailbreak
prompts, protecting LLMs from such malicious attacks remains an important and
challenging task. In this paper, we propose QGuard, a simple yet effective
safety guard method, that utilizes question prompting to block harmful prompts
in a zero-shot manner. Our method can defend LLMs not only from text-based
harmful prompts but also from multi-modal harmful prompt attacks. Moreover, by
diversifying and modifying guard questions, our approach remains robust against
the latest harmful prompts without fine-tuning. Experimental results show that
our model performs competitively on both text-only and multi-modal harmful
datasets. Additionally, by providing an analysis of question prompting, we
enable a white-box analysis of user inputs. We believe our method provides
valuable insights for real-world LLM services in mitigating security risks
associated with harmful prompts.
QGuard, a safety guard method using question prompting, effectively defends LLMs against harmful and multi-modal malicious prompts without fine-tuning.
Large Language Models LLMs harmful prompts jailbreak prompts safety guard question prompting multi-modal harmful prompts