Cooperative Inference of Diffusion Models for Federated Synthetic Data Generation
Seok-Ju Hahn, Junghye Lee
Synthetic data generation is an appealing solution for a data-centric and privacy-preserving federated learning system. However, it is also appalling to adopt in practice, due to the high computational and communication costs of generative models, such as diffusion models. To reconcile these challenges, we define the task of collaborative synthetic data generation as a cooperative sampling procedure from the mixture of decentralized distributions. We propose to implement this by leveraging the composition-friendly nature of diffusion models, building upon a theoretically grounded step-size condition of the unadjusted Langevin algorithm that ensures both compatibility with diffusion sampling and convergence to the target density. As a result, we can directly obtain refined synthetic data across clients, without exchanging model parameters. We empirically demonstrate that our framework reduces communication overhead while significantly improving generation quality, even with differential privacy guarantees.