Do LLMs Have Political Correctness? Analyzing Ethical Biases and
  Jailbreak Vulnerabilities in AI Systems
Although large language models (LLMs) demonstrate impressive proficiency in
various tasks, they present potential safety risks, such as `jailbreaks', where
malicious inputs can coerce LLMs into generating harmful content. To address
these issues, many LLM developers have implemented various safety measures to
align these models. This alignment involves several techniques, including data
filtering during pre-training, supervised fine-tuning, reinforcement learning
from human feedback, and red-teaming exercises. These methods often introduce
deliberate and intentional biases similar to Political Correctness (PC) to
ensure the ethical behavior of LLMs. In this paper, we delve into the
intentional biases injected into LLMs for safety purposes and examine methods
to circumvent these safety alignment techniques. Notably, these intentional
biases result in a jailbreaking success rate in GPT-4o models that differs by
20% between non-binary and cisgender keywords and by 16% between white and
black keywords, even when the other parts of the prompts are identical. We
introduce the concept of PCJailbreak, highlighting the inherent risks posed by
these safety-induced biases. Additionally, we propose an efficient defense
method PCDefense, which prevents jailbreak attempts by injecting defense
prompts prior to generation. PCDefense stands as an appealing alternative to
Guard Models, such as Llama-Guard, that require additional inference cost after
text generation. Our findings emphasize the urgent need for LLM developers to
adopt a more responsible approach when designing and implementing safety
measures.
671210b6d86855660a264b96
Isack Lee
claimed_verified
2024-10-21T07:48:12.573Z
655c2f00360e4f90cb72cd90
/avatars/b1c617eeeb24eeae8c4c0c6884534544.svg
lee isack
user
toast224
671210b6d86855660a264b97
Haebin Seong
extracted_confirmed
2024-10-18T07:39:40.919Z
63a9379e2e05ca32e352d93b
/avatars/6cda37befc873a92ed6d5dcba507954a.svg
Haebin Seong
user
hbseong
Do LLMs Have Political Correctness? Analyzing Ethical Biases and
  Jailbreak Vulnerabilities in AI Systems
Although large language models (LLMs) demonstrate impressive proficiency in
various tasks, they present potential safety risks, such as `jailbreaks', where
malicious inputs can coerce LLMs into generating harmful content. To address
these issues, many LLM developers have implemented various safety measures to
align these models. This alignment involves several techniques, including data
filtering during pre-training, supervised fine-tuning, reinforcement learning
from human feedback, and red-teaming exercises. These methods often introduce
deliberate and intentional biases similar to Political Correctness (PC) to
ensure the ethical behavior of LLMs. In this paper, we delve into the
intentional biases injected into LLMs for safety purposes and examine methods
to circumvent these safety alignment techniques. Notably, these intentional
biases result in a jailbreaking success rate in GPT-4o models that differs by
20% between non-binary and cisgender keywords and by 16% between white and
black keywords, even when the other parts of the prompts are identical. We
introduce the concept of PCJailbreak, highlighting the inherent risks posed by
these safety-induced biases. Additionally, we propose an efficient defense
method PCDefense, which prevents jailbreak attempts by injecting defense
prompts prior to generation. PCDefense stands as an appealing alternative to
Guard Models, such as Llama-Guard, that require additional inference cost after
text generation. Our findings emphasize the urgent need for LLM developers to
adopt a more responsible approach when designing and implementing safety
measures.
Research explores intentional biases in large language models for safety, highlighting jailbreaking risks and proposing a defense method to mitigate these issues without additional inference cost.
large language models LLMs jailbreaks data filtering supervised fine-tuning reinforcement learning from human feedback red-teaming exercises Political Correctness (PC) jailbreaking success rate non-binary cisgender white black PCJailbreak PCDefense Guard Models Llama-Guard