Unveiling Safety Vulnerabilities of Large Language Models
As large language models become more prevalent, their possible harmful or
inappropriate responses are a cause for concern. This paper introduces a unique
dataset containing adversarial examples in the form of questions, which we call
AttaQ, designed to provoke such harmful or inappropriate responses. We assess
the efficacy of our dataset by analyzing the vulnerabilities of various models
when subjected to it. Additionally, we introduce a novel automatic approach for
identifying and naming vulnerable semantic regions - input semantic areas for
which the model is likely to produce harmful outputs. This is achieved through
the application of specialized clustering techniques that consider both the
semantic similarity of the input attacks and the harmfulness of the model's
responses. Automatically identifying vulnerable semantic regions enhances the
evaluation of model weaknesses, facilitating targeted improvements to its
safety mechanisms and overall reliability.
654afb307ed5797e22f36c01
George Kour
admin_assigned
2023-11-08T09:12:02.482Z
64540de813dba495b2e0354e
https://cdn-avatars.huggingface.co/v1/production/uploads/64540de813dba495b2e0354e/w87XdCBRpNw6Kd2xn4AkY.png
George Kour
user
gkour
654afb307ed5797e22f36c02
Marcel Zalmanovici
admin_assigned
2023-11-08T09:12:15.164Z
64bfc29c5ce3d382c05bbe8f
/avatars/5f4e6659350e20a27d19b0fb4e110bc9.svg
Marcel Zalmanovici
user
m-ibm
654afb307ed5797e22f36c03
Naama Zwerdling
654afb307ed5797e22f36c04
Esther Goldbraich
admin_assigned
2023-11-08T09:12:25.672Z
62a5e10bb865939649124ae8
/avatars/3e22777365ced0f37dfb675a934d71ec.svg
Esther Goldbraich
user
esthergold
654afb307ed5797e22f36c05
Ora Nova Fandina
claimed_verified
2023-11-08T15:39:52.356Z
648c5091a47a0850f947e7c3
/avatars/580eeee0e0a07ea10a0d8bb87f5a829c.svg
Ora Fandina
user
orafandina
654afb307ed5797e22f36c06
Ateret Anaby-Tavor
654afb307ed5797e22f36c07
Orna Raz
654afb307ed5797e22f36c08
Eitan Farchi
Unveiling Safety Vulnerabilities of Large Language Models
As large language models become more prevalent, their possible harmful or
inappropriate responses are a cause for concern. This paper introduces a unique
dataset containing adversarial examples in the form of questions, which we call
AttaQ, designed to provoke such harmful or inappropriate responses. We assess
the efficacy of our dataset by analyzing the vulnerabilities of various models
when subjected to it. Additionally, we introduce a novel automatic approach for
identifying and naming vulnerable semantic regions - input semantic areas for
which the model is likely to produce harmful outputs. This is achieved through
the application of specialized clustering techniques that consider both the
semantic similarity of the input attacks and the harmfulness of the model's
responses. Automatically identifying vulnerable semantic regions enhances the
evaluation of model weaknesses, facilitating targeted improvements to its
safety mechanisms and overall reliability.
A dataset called AttaQ is introduced to identify and evaluate vulnerable semantic regions in large language models, using adversarial questions to detect harmful responses and improve model safety.
adversarial examples AttaQ automatic approach vulnerable semantic regions specialized clustering techniques semantic similarity harmful outputs large language models evaluation of model weaknesses targeted improvements safety mechanisms