Interpolating between Images with Diffusion Models
One little-explored frontier of image generation and editing is the task of
interpolating between two input images, a feature missing from all currently
deployed image generation pipelines. We argue that such a feature can expand
the creative applications of such models, and propose a method for zero-shot
interpolation using latent diffusion models. We apply interpolation in the
latent space at a sequence of decreasing noise levels, then perform denoising
conditioned on interpolated text embeddings derived from textual inversion and
(optionally) subject poses. For greater consistency, or to specify additional
criteria, we can generate several candidates and use CLIP to select the highest
quality image. We obtain convincing interpolations across diverse subject
poses, image styles, and image content, and show that standard quantitative
metrics such as FID are insufficient to measure the quality of an
interpolation. Code and data are available at
https://clintonjwang.github.io/interpolation.
64bf855486e7fb5b8aaa2989
Clinton J. Wang
admin_assigned
2023-07-25T09:10:42.853Z
6260d9038144532e6ff059f3
/avatars/61b9a140db8757d8b2f76512609cf3cc.svg
Clinton Wang
user
clintonjwang
64bf855486e7fb5b8aaa298a
Polina Golland
Interpolating between Images with Diffusion Models
One little-explored frontier of image generation and editing is the task of
interpolating between two input images, a feature missing from all currently
deployed image generation pipelines. We argue that such a feature can expand
the creative applications of such models, and propose a method for zero-shot
interpolation using latent diffusion models. We apply interpolation in the
latent space at a sequence of decreasing noise levels, then perform denoising
conditioned on interpolated text embeddings derived from textual inversion and
(optionally) subject poses. For greater consistency, or to specify additional
criteria, we can generate several candidates and use CLIP to select the highest
quality image. We obtain convincing interpolations across diverse subject
poses, image styles, and image content, and show that standard quantitative
metrics such as FID are insufficient to measure the quality of an
interpolation. Code and data are available at
https://clintonjwang.github.io/interpolation.
Latent diffusion models enable zero-shot image interpolation by denoising in latent space with text embeddings and pose conditions, producing high-quality interpolations across varied subjects, styles, and content.
latent diffusion models latent space denoising interpolated text embeddings textual inversion subject poses CLIP FID