Speak Easy: Eliciting Harmful Jailbreaks from LLMs with Simple
  Interactions
Despite extensive safety alignment efforts, large language models (LLMs)
remain vulnerable to jailbreak attacks that elicit harmful behavior. While
existing studies predominantly focus on attack methods that require technical
expertise, two critical questions remain underexplored: (1) Are jailbroken
responses truly useful in enabling average users to carry out harmful actions?
(2) Do safety vulnerabilities exist in more common, simple human-LLM
interactions? In this paper, we demonstrate that LLM responses most effectively
facilitate harmful actions when they are both actionable and informative--two
attributes easily elicited in multi-step, multilingual interactions. Using this
insight, we propose HarmScore, a jailbreak metric that measures how effectively
an LLM response enables harmful actions, and Speak Easy, a simple multi-step,
multilingual attack framework. Notably, by incorporating Speak Easy into direct
request and jailbreak baselines, we see an average absolute increase of 0.319
in Attack Success Rate and 0.426 in HarmScore in both open-source and
proprietary LLMs across four safety benchmarks. Our work reveals a critical yet
often overlooked vulnerability: Malicious users can easily exploit common
interaction patterns for harmful intentions.
67a5a9357415f9155e9b4b58
Yik Siu Chan
67a5a9357415f9155e9b4b59
Narutatsu Ri
claimed_verified
2025-02-07T09:57:58.519Z
64698ed0dcbb937d56b9dd02
/avatars/407afd70356f9d0669d4e2d39b9a8740.svg
Narutatsu Ri
user
narutatsuri
67a5a9357415f9155e9b4b5a
Yuxin Xiao
claimed_verified
2025-02-07T09:58:00.910Z
64bf072bae436c8813494ba3
/avatars/afb96d2bbf90411f4b1a030ebebff300.svg
Yuxin Xiao
user
YuxinXiao
67a5a9357415f9155e9b4b5b
Marzyeh Ghassemi
Speak Easy: Eliciting Harmful Jailbreaks from LLMs with Simple
  Interactions
Despite extensive safety alignment efforts, large language models (LLMs)
remain vulnerable to jailbreak attacks that elicit harmful behavior. While
existing studies predominantly focus on attack methods that require technical
expertise, two critical questions remain underexplored: (1) Are jailbroken
responses truly useful in enabling average users to carry out harmful actions?
(2) Do safety vulnerabilities exist in more common, simple human-LLM
interactions? In this paper, we demonstrate that LLM responses most effectively
facilitate harmful actions when they are both actionable and informative--two
attributes easily elicited in multi-step, multilingual interactions. Using this
insight, we propose HarmScore, a jailbreak metric that measures how effectively
an LLM response enables harmful actions, and Speak Easy, a simple multi-step,
multilingual attack framework. Notably, by incorporating Speak Easy into direct
request and jailbreak baselines, we see an average absolute increase of 0.319
in Attack Success Rate and 0.426 in HarmScore in both open-source and
proprietary LLMs across four safety benchmarks. Our work reveals a critical yet
often overlooked vulnerability: Malicious users can easily exploit common
interaction patterns for harmful intentions.
HarmScore and Speak Easy reveal that common interaction patterns in large language models can facilitate harmful actions, increasing attack success and harm scores.
LLMs jailbreak attacks harmful behavior HarmScore Speak Easy Attack Success Rate HarmScore