Training-Free Safe Text Embedding Guidance for Text-to-Image Diffusion Models
Byeonghu Na, Mina Kang, Jiseok Kwak, Minsang Park, Jiwoo Shin, SeJoon Jun, Gayoung Lee, Jin-Hwa Kim, Il-chul Moon
Text-to-image models have recently made significant advances in generating realistic and semantically accurate images, driven by advanced diffusion models and large-scale web-crawled datasets. However, these datasets often contain inappropriate or biased content, raising concerns about the generation of harmful outputs when provided with malicious input text prompts. In this paper, we propose Safe Text embedding Guidance (STG), a training-free approach to improve the safety of text-to-image diffusion models by guiding the text embeddings during the sampling process. STG adjusts the text embeddings based on a safety function evaluated on the expected final denoised image, allowing the model to generate safer outputs without additional training. Theoretically, we show that STG attempts to align the underlying model distribution with desired safety constraints, thereby achieving safer outputs while minimizing degradation of the original generative quality. We experimentally validate STG on various safety scenarios, including nudity, violence, and artist-style removal. We show that STG consistently outperforms both training-based and training-free baselines in removing unsafe content while preserving the core semantic intent of input prompts.