Semantic Surgery: Zero-Shot Concept Erasure in Diffusion Models
Lexiang Xiong, Liu Chengyu, Jingwen Ye, YAN LIU, Yuecong Xu
With the growing power of text-to-image diffusion models, their potential to generate harmful or biased content has become a pressing concern, motivating the development of concept erasure techniques. Existing approaches, whether relying on retraining or not, frequently compromise the generative capabilities of the target model in achieving concept erasure, often struggling with a critical trade-off between erasure completeness and the preservation of unrelated content (locality). Here, we introduce Semantic Surgery, a novel training-free framework for zero-shot concept erasure. Semantic Surgery directly operates on text embeddings before the diffusion process, aiming to neutralize undesired concepts at their semantic origin and thereby enhance both erasure completeness and the locality of generation by modifying the global semantic input to the diffusion model. Specifically, Semantic Surgery dynamically estimates the presence and intensity of target concepts within an input prompt's global semantics, based on which it performs a calibrated, scaled vector subtraction from the entire text embedding to neutralize their influence at the source. The overall framework consists of a Co-Occurrence Encoding module for robust multi-concept erasure by considering their joint semantic signatures, and an optional visual feedback loop that refines the textual embedding to address Latent Concept Persistence, thereby reinforcing erasure throughout the subsequent denoising process. Our proposed Semantic Surgery requires no model retraining and adapts dynamically to the specific concepts and their intensity detected in each input prompt, ensuring precise and context-aware interventions. Extensive experiments are conducted on object, explicit content, artistic style, and multi-celebrity erasure tasks, demonstrating that our method significantly outperforms state-of-the-art approaches. That is, our proposed concept erasure framework achieves superior completeness and robustness while preserving locality and general image quality, thereby offering an effective and practical solution in text-to-image generation.