Locality in Image Diffusion Models Emerges from Data Statistics
Artem Lukoianov, Chenyang Yuan, Justin Solomon, Vincent Sitzmann
Among generative models, diffusion models are uniquely intriguing due to the existence of a closed-form Bayes-optimal solution to their training objective, often referred to as the optimal denoiser.However, this optimal denoiser merely reproduces images in the training set, and hence fails to serve as a complete theory for the behavior of deep diffusion models. Recent work has attempted to characterize this gap between the optimal denoiser and deep diffusion models and has produced analytical, training-free models that can generate images that resemble those generated by a trained UNet. The best-performing method suggests that shift equivariance and locality inductive biases of convolutional neural networks are the cause of this gap, and hence, incorporates these assumptions into its analytical model. In this work, we present evidence that the locality in deep diffusion models emerges as a statistical property of the image dataset, \emph{not} due to the inductive bias of convolutional neural networks. Specifically, we demonstrate that an optimal parametric linear denoiser exhibits similar locality properties and provides a theoretical analysis that grounds this in a principal component analysis of the training set.We then show that an analytical denoiser based on these statistics better matches scores predicted by a deep diffusion model than the prior, expert-crafted alternative.