Generalizable Origin Identification for Text-Guided Image-to-Image
  Diffusion Models
Text-guided image-to-image diffusion models excel in translating images based
on textual prompts, allowing for precise and creative visual modifications.
However, such a powerful technique can be misused for spreading misinformation,
infringing on copyrights, and evading content tracing. This motivates us to
introduce the task of origin IDentification for text-guided Image-to-image
Diffusion models (ID^2), aiming to retrieve the original image of a given
translated query. A straightforward solution to ID^2 involves training a
specialized deep embedding model to extract and compare features from both
query and reference images. However, due to visual discrepancy across
generations produced by different diffusion models, this similarity-based
approach fails when training on images from one model and testing on those from
another, limiting its effectiveness in real-world applications. To solve this
challenge of the proposed ID^2 task, we contribute the first dataset and a
theoretically guaranteed method, both emphasizing generalizability. The curated
dataset, OriPID, contains abundant Origins and guided Prompts, which can be
used to train and test potential IDentification models across various diffusion
models. In the method section, we first prove the existence of a linear
transformation that minimizes the distance between the pre-trained Variational
Autoencoder (VAE) embeddings of generated samples and their origins.
Subsequently, it is demonstrated that such a simple linear transformation can
be generalized across different diffusion models. Experimental results show
that the proposed method achieves satisfying generalization performance,
significantly surpassing similarity-based methods (+31.6% mAP), even those
with generalization designs.
677f28c33d2765f5655e948f
Wenhao Wang
claimed_verified
2025-01-13T08:54:38.907Z
62b32a4429a410b7f6b06710
https://cdn-avatars.huggingface.co/v1/production/uploads/62b32a4429a410b7f6b06710/VzgvmnlYZWuifZTkIkCxy.jpeg
Wenhao Wang
user
WenhaoWang
677f28c33d2765f5655e9490
Yifan Sun
677f28c33d2765f5655e9491
Zongxin Yang
677f28c33d2765f5655e9492
Zhentao Tan
677f28c33d2765f5655e9493
Zhengdong Hu
677f28c33d2765f5655e9494
Yi Yang
Generalizable Origin Identification for Text-Guided Image-to-Image
  Diffusion Models
Text-guided image-to-image diffusion models excel in translating images based
on textual prompts, allowing for precise and creative visual modifications.
However, such a powerful technique can be misused for spreading misinformation,
infringing on copyrights, and evading content tracing. This motivates us to
introduce the task of origin IDentification for text-guided Image-to-image
Diffusion models (ID^2), aiming to retrieve the original image of a given
translated query. A straightforward solution to ID^2 involves training a
specialized deep embedding model to extract and compare features from both
query and reference images. However, due to visual discrepancy across
generations produced by different diffusion models, this similarity-based
approach fails when training on images from one model and testing on those from
another, limiting its effectiveness in real-world applications. To solve this
challenge of the proposed ID^2 task, we contribute the first dataset and a
theoretically guaranteed method, both emphasizing generalizability. The curated
dataset, OriPID, contains abundant Origins and guided Prompts, which can be
used to train and test potential IDentification models across various diffusion
models. In the method section, we first prove the existence of a linear
transformation that minimizes the distance between the pre-trained Variational
Autoencoder (VAE) embeddings of generated samples and their origins.
Subsequently, it is demonstrated that such a simple linear transformation can
be generalized across different diffusion models. Experimental results show
that the proposed method achieves satisfying generalization performance,
significantly surpassing similarity-based methods (+31.6% mAP), even those
with generalization designs.
A method for origin identification in text-guided image-to-image diffusion models is introduced, achieving superior generalization performance compared to similarity-based approaches.
text-guided image-to-image diffusion models origin identification deep embedding model visual discrepancy ID$^2$ OriPID dataset linear transformation Variational Autoencoder (VAE) generalization performance mAP