JailDAM: Jailbreak Detection with Adaptive Memory for Vision-Language
  Model
Multimodal large language models (MLLMs) excel in vision-language tasks but
also pose significant risks of generating harmful content, particularly through
jailbreak attacks. Jailbreak attacks refer to intentional manipulations that
bypass safety mechanisms in models, leading to the generation of inappropriate
or unsafe content. Detecting such attacks is critical to ensuring the
responsible deployment of MLLMs. Existing jailbreak detection methods face
three primary challenges: (1) Many rely on model hidden states or gradients,
limiting their applicability to white-box models, where the internal workings
of the model are accessible; (2) They involve high computational overhead from
uncertainty-based analysis, which limits real-time detection, and (3) They
require fully labeled harmful datasets, which are often scarce in real-world
settings. To address these issues, we introduce a test-time adaptive framework
called JAILDAM. Our method leverages a memory-based approach guided by
policy-driven unsafe knowledge representations, eliminating the need for
explicit exposure to harmful data. By dynamically updating unsafe knowledge
during test-time, our framework improves generalization to unseen jailbreak
strategies while maintaining efficiency. Experiments on multiple VLM jailbreak
benchmarks demonstrate that JAILDAM delivers state-of-the-art performance in
harmful content detection, improving both accuracy and speed.
67f4a80a261cb1c328d9b0a2
Yi Nian
67f4a80a261cb1c328d9b0a3
Shenzhe Zhu
claimed_verified
2025-04-08T06:51:27.601Z
6614243f67d7bfc73afc6b77
https://cdn-avatars.huggingface.co/v1/production/uploads/6614243f67d7bfc73afc6b77/ifdjAb_BPraEJ2eIySk_K.jpeg
Shenzhe Zhu
user
Chouoftears
67f4a80a261cb1c328d9b0a4
Yuehan Qin
admin_assigned
2025-04-08T10:46:00.895Z
65e56670f7daaa71c1b9ee5a
/avatars/0ff27979dd756b6bb63818ba49fcfec5.svg
Yuehan Qin
user
yuehanqi
67f4a80a261cb1c328d9b0a5
Li Li
67f4a80a261cb1c328d9b0a6
Ziyi Wang
67f4a80a261cb1c328d9b0a7
Chaowei Xiao
67f4a80a261cb1c328d9b0a8
Yue Zhao
admin_assigned
2025-04-08T10:46:42.241Z
65e56670f7daaa71c1b9ee5a
/avatars/0ff27979dd756b6bb63818ba49fcfec5.svg
Yuehan Qin
user
yuehanqi
JailDAM: Jailbreak Detection with Adaptive Memory for Vision-Language
  Model
Multimodal large language models (MLLMs) excel in vision-language tasks but
also pose significant risks of generating harmful content, particularly through
jailbreak attacks. Jailbreak attacks refer to intentional manipulations that
bypass safety mechanisms in models, leading to the generation of inappropriate
or unsafe content. Detecting such attacks is critical to ensuring the
responsible deployment of MLLMs. Existing jailbreak detection methods face
three primary challenges: (1) Many rely on model hidden states or gradients,
limiting their applicability to white-box models, where the internal workings
of the model are accessible; (2) They involve high computational overhead from
uncertainty-based analysis, which limits real-time detection, and (3) They
require fully labeled harmful datasets, which are often scarce in real-world
settings. To address these issues, we introduce a test-time adaptive framework
called JAILDAM. Our method leverages a memory-based approach guided by
policy-driven unsafe knowledge representations, eliminating the need for
explicit exposure to harmful data. By dynamically updating unsafe knowledge
during test-time, our framework improves generalization to unseen jailbreak
strategies while maintaining efficiency. Experiments on multiple VLM jailbreak
benchmarks demonstrate that JAILDAM delivers state-of-the-art performance in
harmful content detection, improving both accuracy and speed.
JAILDAM is a test-time adaptive framework that detects jailbreak attacks in multimodal large language models without requiring labeled harmful data, improving accuracy and efficiency.
multimodal large language models jailbreak attacks safety mechanisms hidden states gradients white-box models uncertainty-based analysis memory-based approach policy-driven unsafe knowledge representations harmful content detection VLM jailbreak benchmarks