Cross-Modality Safety Alignment
As Artificial General Intelligence (AGI) becomes increasingly integrated into
various facets of human life, ensuring the safety and ethical alignment of such
systems is paramount. Previous studies primarily focus on single-modality
threats, which may not suffice given the integrated and complex nature of
cross-modality interactions. We introduce a novel safety alignment challenge
called Safe Inputs but Unsafe Output (SIUO) to evaluate cross-modality safety
alignment. Specifically, it considers cases where single modalities are safe
independently but could potentially lead to unsafe or unethical outputs when
combined. To empirically investigate this problem, we developed the SIUO, a
cross-modality benchmark encompassing 9 critical safety domains, such as
self-harm, illegal activities, and privacy violations. Our findings reveal
substantial safety vulnerabilities in both closed- and open-source LVLMs, such
as GPT-4V and LLaVA, underscoring the inadequacy of current models to reliably
interpret and respond to complex, real-world scenarios.
667aa83869c3e097d596de29
Siyin Wang
claimed_verified
2024-06-25T13:34:23.735Z
64c3c631e77ea9f28111172a
/avatars/495dbb73b69c399bae780da3118e332f.svg
Siyin Wang
user
sinwang
667aa83869c3e097d596de2a
Xingsong Ye
667aa83869c3e097d596de2b
Qinyuan Cheng
claimed_verified
2025-06-24T08:15:37.427Z
63ec4715c81b6a52391c46b8
/avatars/496819b5075a1a834a2b9edeb068c80e.svg
QinyuanCheng
user
Cqy2019
667aa83869c3e097d596de2c
Junwen Duan
667aa83869c3e097d596de2d
Shimin Li
667aa83869c3e097d596de2e
Jinlan Fu
667aa83869c3e097d596de2f
Xipeng Qiu
667aa83869c3e097d596de30
Xuanjing Huang
Cross-Modality Safety Alignment
As Artificial General Intelligence (AGI) becomes increasingly integrated into
various facets of human life, ensuring the safety and ethical alignment of such
systems is paramount. Previous studies primarily focus on single-modality
threats, which may not suffice given the integrated and complex nature of
cross-modality interactions. We introduce a novel safety alignment challenge
called Safe Inputs but Unsafe Output (SIUO) to evaluate cross-modality safety
alignment. Specifically, it considers cases where single modalities are safe
independently but could potentially lead to unsafe or unethical outputs when
combined. To empirically investigate this problem, we developed the SIUO, a
cross-modality benchmark encompassing 9 critical safety domains, such as
self-harm, illegal activities, and privacy violations. Our findings reveal
substantial safety vulnerabilities in both closed- and open-source LVLMs, such
as GPT-4V and LLaVA, underscoring the inadequacy of current models to reliably
interpret and respond to complex, real-world scenarios.
A novel challenge, SIUO, evaluates the safety of cross-modality interactions in large vision-language models, revealing significant vulnerabilities in their ability to handle real-world scenarios.
Safe Inputs but Unsafe Output SIUO cross-modality safety alignment LLMs LVLMs GPT-4V LLaVA