DiffMorpher: Unleashing the Capability of Diffusion Models for Image
  Morphing
Diffusion models have achieved remarkable image generation quality surpassing
previous generative models. However, a notable limitation of diffusion models,
in comparison to GANs, is their difficulty in smoothly interpolating between
two image samples, due to their highly unstructured latent space. Such a smooth
interpolation is intriguing as it naturally serves as a solution for the image
morphing task with many applications. In this work, we present DiffMorpher, the
first approach enabling smooth and natural image interpolation using diffusion
models. Our key idea is to capture the semantics of the two images by fitting
two LoRAs to them respectively, and interpolate between both the LoRA
parameters and the latent noises to ensure a smooth semantic transition, where
correspondence automatically emerges without the need for annotation. In
addition, we propose an attention interpolation and injection technique and a
new sampling schedule to further enhance the smoothness between consecutive
images. Extensive experiments demonstrate that DiffMorpher achieves starkly
better image morphing effects than previous methods across a variety of object
categories, bridging a critical functional gap that distinguished diffusion
models from GANs.
657925a04d989b0a68bfbafe
Kaiwen Zhang
admin_assigned
2023-12-13T08:36:55.843Z
641087ddf52d7eb22e0673dd
/avatars/946f29569aeda2cbd9ef2b94b1ab1391.svg
Kevin Zhang
user
Kevin-thu
657925a04d989b0a68bfbaff
Yifan Zhou
claimed_verified
2024-12-02T12:52:47.947Z
64a63f9449b08110f761cd73
/avatars/61860202fc818b105ef24e74dd4f7d3c.svg
Yifan Zhou
user
SingleZombie
657925a04d989b0a68bfbb00
Xudong Xu
657925a04d989b0a68bfbb01
Xingang Pan
657925a04d989b0a68bfbb02
Bo Dai
admin_assigned
2023-12-13T08:40:01.097Z
635f93577c05eb9f59966209
/avatars/add48d7e3790a6b500d6c451ef8b0f75.svg
Intelligent Digital Creation
user
BoDai
DiffMorpher: Unleashing the Capability of Diffusion Models for Image
  Morphing
Diffusion models have achieved remarkable image generation quality surpassing
previous generative models. However, a notable limitation of diffusion models,
in comparison to GANs, is their difficulty in smoothly interpolating between
two image samples, due to their highly unstructured latent space. Such a smooth
interpolation is intriguing as it naturally serves as a solution for the image
morphing task with many applications. In this work, we present DiffMorpher, the
first approach enabling smooth and natural image interpolation using diffusion
models. Our key idea is to capture the semantics of the two images by fitting
two LoRAs to them respectively, and interpolate between both the LoRA
parameters and the latent noises to ensure a smooth semantic transition, where
correspondence automatically emerges without the need for annotation. In
addition, we propose an attention interpolation and injection technique and a
new sampling schedule to further enhance the smoothness between consecutive
images. Extensive experiments demonstrate that DiffMorpher achieves starkly
better image morphing effects than previous methods across a variety of object
categories, bridging a critical functional gap that distinguished diffusion
models from GANs.
DiffMorpher enhances smooth image interpolation using diffusion models by capturing image semantics with LoRA and applying attention interpolation techniques.
diffusion models GANs image interpolation LoRA attention interpolation latent noises image morphing