When Good Sounds Go Adversarial: Jailbreaking Audio-Language Models with
  Benign Inputs
As large language models become increasingly integrated into daily life,
audio has emerged as a key interface for human-AI interaction. However, this
convenience also introduces new vulnerabilities, making audio a potential
attack surface for adversaries. Our research introduces WhisperInject, a
two-stage adversarial audio attack framework that can manipulate
state-of-the-art audio language models to generate harmful content. Our method
uses imperceptible perturbations in audio inputs that remain benign to human
listeners. The first stage uses a novel reward-based optimization method,
Reinforcement Learning with Projected Gradient Descent (RL-PGD), to guide the
target model to circumvent its own safety protocols and generate harmful native
responses. This native harmful response then serves as the target for Stage 2,
Payload Injection, where we use Projected Gradient Descent (PGD) to optimize
subtle perturbations that are embedded into benign audio carriers, such as
weather queries or greeting messages. Validated under the rigorous
StrongREJECT, LlamaGuard, as well as Human Evaluation safety evaluation
framework, our experiments demonstrate a success rate exceeding 86% across
Qwen2.5-Omni-3B, Qwen2.5-Omni-7B, and Phi-4-Multimodal. Our work demonstrates a
new class of practical, audio-native threats, moving beyond theoretical
exploits to reveal a feasible and covert method for manipulating AI behavior.
6892ae738da45ffb0a2b23ca
Bodam Kim
claimed_verified
2025-09-12T16:22:42.628Z
654e84e5298f66afe8a5ebd6
/avatars/5c49c5724b649c787c1d861f06bd078a.svg
Bodam Kim
user
bodam
6892ae738da45ffb0a2b23cb
Hiskias Dingeto
claimed_verified
2025-08-13T07:28:21.803Z
643fd9e2e44f30a72321648c
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/Hmcnp0xtcQZz3eGBmDt5O.png
Hisakias Dingeto
user
hisku
6892ae738da45ffb0a2b23cc
Taeyoun Kwon
claimed_verified
2025-08-25T08:28:47.900Z
687d34e1fb7cc11a14428935
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/FYzQKblvrbU082pIm2xDG.png
TaeyounKwon
user
taeyoun811
6892ae738da45ffb0a2b23cd
Dasol Choi
6892ae738da45ffb0a2b23ce
DongGeon Lee
claimed_verified
2025-08-06T19:17:41.907Z
6540fbf9cb7fffd683942b43
https://cdn-avatars.huggingface.co/v1/production/uploads/6540fbf9cb7fffd683942b43/7o4Pwz7dLjUeI1nbV57Fw.png
DongGeon Lee
user
oneonlee
6892ae738da45ffb0a2b23cf
Haon Park
6892ae738da45ffb0a2b23d0
JaeHoon Lee
6892ae738da45ffb0a2b23d1
Jongho Shin
When Good Sounds Go Adversarial: Jailbreaking Audio-Language Models with
  Benign Inputs
As large language models become increasingly integrated into daily life,
audio has emerged as a key interface for human-AI interaction. However, this
convenience also introduces new vulnerabilities, making audio a potential
attack surface for adversaries. Our research introduces WhisperInject, a
two-stage adversarial audio attack framework that can manipulate
state-of-the-art audio language models to generate harmful content. Our method
uses imperceptible perturbations in audio inputs that remain benign to human
listeners. The first stage uses a novel reward-based optimization method,
Reinforcement Learning with Projected Gradient Descent (RL-PGD), to guide the
target model to circumvent its own safety protocols and generate harmful native
responses. This native harmful response then serves as the target for Stage 2,
Payload Injection, where we use Projected Gradient Descent (PGD) to optimize
subtle perturbations that are embedded into benign audio carriers, such as
weather queries or greeting messages. Validated under the rigorous
StrongREJECT, LlamaGuard, as well as Human Evaluation safety evaluation
framework, our experiments demonstrate a success rate exceeding 86% across
Qwen2.5-Omni-3B, Qwen2.5-Omni-7B, and Phi-4-Multimodal. Our work demonstrates a
new class of practical, audio-native threats, moving beyond theoretical
exploits to reveal a feasible and covert method for manipulating AI behavior.
WhisperInject uses RL-PGD and PGD to create imperceptible audio perturbations that manipulate large language models into generating harmful content.
WhisperInject Reinforcement Learning with Projected Gradient Descent (RL-PGD) Projected Gradient Descent (PGD) audio language models safety protocols StrongREJECT LlamaGuard Human Evaluation Qwen2.5-Omni-3B Qwen2.5-Omni-7B Phi-4-Multimodal