Understanding Hallucinations in Diffusion Models through Mode
  Interpolation
Colloquially speaking, image generation models based upon diffusion processes
are frequently said to exhibit "hallucinations," samples that could never occur
in the training data. But where do such hallucinations come from? In this
paper, we study a particular failure mode in diffusion models, which we term
mode interpolation. Specifically, we find that diffusion models smoothly
"interpolate" between nearby data modes in the training set, to generate
samples that are completely outside the support of the original training
distribution; this phenomenon leads diffusion models to generate artifacts that
never existed in real data (i.e., hallucinations). We systematically study the
reasons for, and the manifestation of this phenomenon. Through experiments on
1D and 2D Gaussians, we show how a discontinuous loss landscape in the
diffusion model's decoder leads to a region where any smooth approximation will
cause such hallucinations. Through experiments on artificial datasets with
various shapes, we show how hallucination leads to the generation of
combinations of shapes that never existed. Finally, we show that diffusion
models in fact know when they go out of support and hallucinate. This is
captured by the high variance in the trajectory of the generated sample towards
the final few backward sampling process. Using a simple metric to capture this
variance, we can remove over 95% of hallucinations at generation time while
retaining 96% of in-support samples. We conclude our exploration by showing the
implications of such hallucination (and its removal) on the collapse (and
stabilization) of recursive training on synthetic data with experiments on
MNIST and 2D Gaussians dataset. We release our code at
https://github.com/locuslab/diffusion-model-hallucination.
666cc6a4c0f3d5afc216a089
Sumukh K Aithal
admin_assigned
2024-06-15T09:18:51.996Z
6660c0b74228284ed72fc818
/avatars/befaf728c4e517c88adc30536a29d90d.svg
Sumukh Aithal
user
sumukhaithal6
666cc6a4c0f3d5afc216a08a
Pratyush Maini
admin_assigned
2024-06-15T09:18:57.997Z
632c826d1d303f5f9acf5917
/avatars/defca406f2c4f027a7b1be9a5ebbbd01.svg
Pratyush Maini
user
pratyushmaini
666cc6a4c0f3d5afc216a08b
Zachary C. Lipton
666cc6a4c0f3d5afc216a08c
J. Zico Kolter
admin_assigned
2024-06-15T09:19:13.886Z
6561eb612cd8dac568605da3
/avatars/6d145d7e225b16bde64680ba8c05f856.svg
Zico Kolter
user
zkolter
Understanding Hallucinations in Diffusion Models through Mode
  Interpolation
Colloquially speaking, image generation models based upon diffusion processes
are frequently said to exhibit "hallucinations," samples that could never occur
in the training data. But where do such hallucinations come from? In this
paper, we study a particular failure mode in diffusion models, which we term
mode interpolation. Specifically, we find that diffusion models smoothly
"interpolate" between nearby data modes in the training set, to generate
samples that are completely outside the support of the original training
distribution; this phenomenon leads diffusion models to generate artifacts that
never existed in real data (i.e., hallucinations). We systematically study the
reasons for, and the manifestation of this phenomenon. Through experiments on
1D and 2D Gaussians, we show how a discontinuous loss landscape in the
diffusion model's decoder leads to a region where any smooth approximation will
cause such hallucinations. Through experiments on artificial datasets with
various shapes, we show how hallucination leads to the generation of
combinations of shapes that never existed. Finally, we show that diffusion
models in fact know when they go out of support and hallucinate. This is
captured by the high variance in the trajectory of the generated sample towards
the final few backward sampling process. Using a simple metric to capture this
variance, we can remove over 95% of hallucinations at generation time while
retaining 96% of in-support samples. We conclude our exploration by showing the
implications of such hallucination (and its removal) on the collapse (and
stabilization) of recursive training on synthetic data with experiments on
MNIST and 2D Gaussians dataset. We release our code at
https://github.com/locuslab/diffusion-model-hallucination.
Diffusion models generate unrealistic samples, termed hallucinations, through mode interpolation due to a discontinuous loss landscape in the decoder, leading to high variance in backward sampling; this can be mitigated with a simple variance metric.
diffusion models mode interpolation data modes training distribution hallucinations loss landscape generated sample variance recursive training