RoMa: A Robust Model Watermarking Scheme for Protecting IP in Diffusion Models
Yingsha Xie, Rui Min, Zeyu Qin, Fei Ma, Li Shen, Fei Yu, Xiaochun Cao
Preserving intellectual property (IP) within a pre-trained diffusion model is critical for protecting the model's copyright and preventing unauthorized model deployment. In this regard, model watermarking is a common practice that embeds traceable information within model weights and allows for further verification, making it a widely applied scheme for model IP protection. Nevertheless, existing watermarking schemes often face challenges due to their vulnerability to fine-tuning, limiting their practical application in general pre-training and fine-tuning paradigms. Inspired by using mode connectivity to analyze model performance between a pair of connected models, in this paper, we investigate watermark vulnerability by leveraging Linear Mode Connectivity (LMC) as a proxy to analyze the fine-tuning dynamics of watermark performance. Our results show that existing watermarked models tend to converge to sharp minima in the loss landscape, making them vulnerable to loss of watermarking information during fine-tuning. Inspired by our findings, we propose RoMa, a **Ro**bust **M**odel w**a**termarking scheme that improves the robustness of watermarks against fine-tuning by enhancing the smoothness along the watermark-connected path between the pre-trained and watermarked models. Specifically, RoMa decomposes watermarking into two components, including *Embedding Functionality*, which preserves reliable watermark detection capability, and *Path-specific Smoothness*, which leverages extra updates guided by the watermark-connected path to enhance robustness. Extensive experiments demonstrate that RoMa significantly improves watermark robustness while maintaining generation quality. Notably, our method requires (32.83 $\times$) more steps to remove the watermark compared to existing baselines.