Investigating Safety Vulnerabilities of Large Audio-Language Models
  Under Speaker Emotional Variations
Large audio-language models (LALMs) extend text-based LLMs with auditory
understanding, offering new opportunities for multimodal applications. While
their perception, reasoning, and task performance have been widely studied,
their safety alignment under paralinguistic variation remains underexplored.
This work systematically investigates the role of speaker emotion. We construct
a dataset of malicious speech instructions expressed across multiple emotions
and intensities, and evaluate several state-of-the-art LALMs. Our results
reveal substantial safety inconsistencies: different emotions elicit varying
levels of unsafe responses, and the effect of intensity is non-monotonic, with
medium expressions often posing the greatest risk. These findings highlight an
overlooked vulnerability in LALMs and call for alignment strategies explicitly
designed to ensure robustness under emotional variation, a prerequisite for
trustworthy deployment in real-world settings.
68f78ebaa7b9b96b2c13edd1
Bo-Han Feng
68f78ebaa7b9b96b2c13edd2
Chien-Feng Liu
68f78ebaa7b9b96b2c13edd3
Yu-Hsuan Li Liang
68f78ebaa7b9b96b2c13edd4
Chih-Kai Yang
claimed_verified
2025-10-21T14:00:09.544Z
646fa3016441111304fec68d
/avatars/923629340f3785ae8c6e52cf3674d5c2.svg
Chih-Kai Yang
user
zenyn
68f78ebaa7b9b96b2c13edd5
Szu-Wei Fu
68f78ebaa7b9b96b2c13edd6
Zhehuai Chen
68f78ebaa7b9b96b2c13edd7
Ke-Han Lu
68f78ebaa7b9b96b2c13edd8
Sung-Feng Huang
68f78ebaa7b9b96b2c13edd9
Chao-Han Huck Yang
68f78ebaa7b9b96b2c13edda
Yu-Chiang Frank Wang
68f78ebaa7b9b96b2c13eddb
Yun-Nung Chen
68f78ebaa7b9b96b2c13eddc
Hung-yi Lee
Investigating Safety Vulnerabilities of Large Audio-Language Models
  Under Speaker Emotional Variations
Large audio-language models (LALMs) extend text-based LLMs with auditory
understanding, offering new opportunities for multimodal applications. While
their perception, reasoning, and task performance have been widely studied,
their safety alignment under paralinguistic variation remains underexplored.
This work systematically investigates the role of speaker emotion. We construct
a dataset of malicious speech instructions expressed across multiple emotions
and intensities, and evaluate several state-of-the-art LALMs. Our results
reveal substantial safety inconsistencies: different emotions elicit varying
levels of unsafe responses, and the effect of intensity is non-monotonic, with
medium expressions often posing the greatest risk. These findings highlight an
overlooked vulnerability in LALMs and call for alignment strategies explicitly
designed to ensure robustness under emotional variation, a prerequisite for
trustworthy deployment in real-world settings.
Research investigates the impact of speaker emotion on the safety of large audio-language models, revealing inconsistencies and vulnerabilities that require targeted alignment strategies.
large audio-language models LALMs text-based LLMs auditory understanding multimodal applications speaker emotion malicious speech instructions unsafe responses alignment strategies robustness