Prompt2Perturb (P2P): Text-Guided Diffusion-Based Adversarial Attacks on
  Breast Ultrasound Images
Deep neural networks (DNNs) offer significant promise for improving breast
cancer diagnosis in medical imaging. However, these models are highly
susceptible to adversarial attacks--small, imperceptible changes that can
mislead classifiers--raising critical concerns about their reliability and
security. Traditional attacks rely on fixed-norm perturbations, misaligning
with human perception. In contrast, diffusion-based attacks require pre-trained
models, demanding substantial data when these models are unavailable, limiting
practical use in data-scarce scenarios. In medical imaging, however, this is
often unfeasible due to the limited availability of datasets. Building on
recent advancements in learnable prompts, we propose Prompt2Perturb (P2P), a
novel language-guided attack method capable of generating meaningful attack
examples driven by text instructions. During the prompt learning phase, our
approach leverages learnable prompts within the text encoder to create subtle,
yet impactful, perturbations that remain imperceptible while guiding the model
towards targeted outcomes. In contrast to current prompt learning-based
approaches, our P2P stands out by directly updating text embeddings, avoiding
the need for retraining diffusion models. Further, we leverage the finding that
optimizing only the early reverse diffusion steps boosts efficiency while
ensuring that the generated adversarial examples incorporate subtle noise, thus
preserving ultrasound image quality without introducing noticeable artifacts.
We show that our method outperforms state-of-the-art attack techniques across
three breast ultrasound datasets in FID and LPIPS. Moreover, the generated
images are both more natural in appearance and more effective compared to
existing adversarial attacks. Our code will be publicly available
https://github.com/yasamin-med/P2P.
675fdfecdd3e9eeed63d8093
Yasamin Medghalchi
admin_assigned
2024-12-16T10:26:08.678Z
650392bb694f6b620fca36e5
/avatars/35d8f1a0b96835bca81457ff6f361de7.svg
yasamin medghalchi
user
yasimed
675fdfecdd3e9eeed63d8094
Moein Heidari
admin_assigned
2024-12-16T10:26:03.231Z
61ba19bf6122a4fd29049371
https://cdn-avatars.huggingface.co/v1/production/uploads/1639586194527-noauth.jpeg
Moein Heidari
user
moein99
675fdfecdd3e9eeed63d8095
Clayton Allard
claimed_verified
2024-12-18T09:53:02.068Z
6761d1c9ea3649f2ad18b09f
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/n7xizu14ObaZHU9Sve-CK.png
Clayton Allard
user
claytonallard
675fdfecdd3e9eeed63d8096
Leonid Sigal
675fdfecdd3e9eeed63d8097
Ilker Hacihaliloglu
Prompt2Perturb (P2P): Text-Guided Diffusion-Based Adversarial Attacks on
  Breast Ultrasound Images
Deep neural networks (DNNs) offer significant promise for improving breast
cancer diagnosis in medical imaging. However, these models are highly
susceptible to adversarial attacks--small, imperceptible changes that can
mislead classifiers--raising critical concerns about their reliability and
security. Traditional attacks rely on fixed-norm perturbations, misaligning
with human perception. In contrast, diffusion-based attacks require pre-trained
models, demanding substantial data when these models are unavailable, limiting
practical use in data-scarce scenarios. In medical imaging, however, this is
often unfeasible due to the limited availability of datasets. Building on
recent advancements in learnable prompts, we propose Prompt2Perturb (P2P), a
novel language-guided attack method capable of generating meaningful attack
examples driven by text instructions. During the prompt learning phase, our
approach leverages learnable prompts within the text encoder to create subtle,
yet impactful, perturbations that remain imperceptible while guiding the model
towards targeted outcomes. In contrast to current prompt learning-based
approaches, our P2P stands out by directly updating text embeddings, avoiding
the need for retraining diffusion models. Further, we leverage the finding that
optimizing only the early reverse diffusion steps boosts efficiency while
ensuring that the generated adversarial examples incorporate subtle noise, thus
preserving ultrasound image quality without introducing noticeable artifacts.
We show that our method outperforms state-of-the-art attack techniques across
three breast ultrasound datasets in FID and LPIPS. Moreover, the generated
images are both more natural in appearance and more effective compared to
existing adversarial attacks. Our code will be publicly available
https://github.com/yasamin-med/P2P.
Prompt2Perturb generates subtle, imperceptible adversarial perturbations for breast ultrasound images using text guidance, outperforming existing techniques in both naturalness and effectiveness.
deep neural networks adversarial attacks diffusion-based attacks pre-trained models learnable prompts prompt learning text encoder text embeddings reverse diffusion FID LPIPS adversarial examples