Training Data Protection with Compositional Diffusion Models
We introduce Compartmentalized Diffusion Models (CDM), a method to train
different diffusion models (or prompts) on distinct data sources and
arbitrarily compose them at inference time. The individual models can be
trained in isolation, at different times, and on different distributions and
domains and can be later composed to achieve performance comparable to a
paragon model trained on all data simultaneously. Furthermore, each model only
contains information about the subset of the data it was exposed to during
training, enabling several forms of training data protection. In particular,
CDMs are the first method to enable both selective forgetting and continual
learning for large-scale diffusion models, as well as allowing serving
customized models based on the user's access rights. CDMs also allow
determining the importance of a subset of the data in generating particular
samples.
64d06afe1720c7a483026706
Aditya Golatkar
64d06afe1720c7a483026707
Alessandro Achille
64d06afe1720c7a483026708
Ashwin Swaminathan
64d06afe1720c7a483026709
Stefano Soatto
Training Data Protection with Compositional Diffusion Models
We introduce Compartmentalized Diffusion Models (CDM), a method to train
different diffusion models (or prompts) on distinct data sources and
arbitrarily compose them at inference time. The individual models can be
trained in isolation, at different times, and on different distributions and
domains and can be later composed to achieve performance comparable to a
paragon model trained on all data simultaneously. Furthermore, each model only
contains information about the subset of the data it was exposed to during
training, enabling several forms of training data protection. In particular,
CDMs are the first method to enable both selective forgetting and continual
learning for large-scale diffusion models, as well as allowing serving
customized models based on the user's access rights. CDMs also allow
determining the importance of a subset of the data in generating particular
samples.
Compartmentalized Diffusion Models (CDM) enable the isolated training and flexible composition of diffusion models on distinct data sources, supporting selective forgetting, continual learning, and personalized model serving.
Compartmentalized Diffusion Models CDM diffusion models prompts data sources inference time training data protection selective forgetting continual learning customized models access rights data subset importance