Diffuse to Choose: Enriching Image Conditioned Inpainting in Latent
  Diffusion Models for Virtual Try-All
As online shopping is growing, the ability for buyers to virtually visualize
products in their settings-a phenomenon we define as "Virtual Try-All"-has
become crucial. Recent diffusion models inherently contain a world model,
rendering them suitable for this task within an inpainting context. However,
traditional image-conditioned diffusion models often fail to capture the
fine-grained details of products. In contrast, personalization-driven models
such as DreamPaint are good at preserving the item's details but they are not
optimized for real-time applications. We present "Diffuse to Choose," a novel
diffusion-based image-conditioned inpainting model that efficiently balances
fast inference with the retention of high-fidelity details in a given reference
item while ensuring accurate semantic manipulations in the given scene content.
Our approach is based on incorporating fine-grained features from the reference
image directly into the latent feature maps of the main diffusion model,
alongside with a perceptual loss to further preserve the reference item's
details. We conduct extensive testing on both in-house and publicly available
datasets, and show that Diffuse to Choose is superior to existing zero-shot
diffusion inpainting methods as well as few-shot diffusion personalization
algorithms like DreamPaint.
65b334dada72b2526d3f9212
Mehmet Saygin Seyfioglu
admin_assigned
2024-01-26T09:25:32.062Z
6359e109d72fc0539e767330
/avatars/a86d541980b56d6532f2bd8078998c74.svg
Mehmet Saygin Seyfioglu
user
aldraus
65b334dada72b2526d3f9213
Karim Bouyarmane
65b334dada72b2526d3f9214
Suren Kumar
admin_assigned
2024-01-26T09:25:46.702Z
657cd73e156fdf03018cfe33
/avatars/46644cec76c7bca338210beeb3235fb5.svg
Suren Kumar
user
SSKumar
65b334dada72b2526d3f9215
Amir Tavanaei
admin_assigned
2024-01-26T09:25:54.074Z
638f644bc67af472d31a93a0
/avatars/b4f2e07ef457789fedcf20a258d8b7b2.svg
Amir Tavanaei
user
tavanaei
65b334dada72b2526d3f9216
Ismail B. Tutar
Diffuse to Choose: Enriching Image Conditioned Inpainting in Latent
  Diffusion Models for Virtual Try-All
As online shopping is growing, the ability for buyers to virtually visualize
products in their settings-a phenomenon we define as "Virtual Try-All"-has
become crucial. Recent diffusion models inherently contain a world model,
rendering them suitable for this task within an inpainting context. However,
traditional image-conditioned diffusion models often fail to capture the
fine-grained details of products. In contrast, personalization-driven models
such as DreamPaint are good at preserving the item's details but they are not
optimized for real-time applications. We present "Diffuse to Choose," a novel
diffusion-based image-conditioned inpainting model that efficiently balances
fast inference with the retention of high-fidelity details in a given reference
item while ensuring accurate semantic manipulations in the given scene content.
Our approach is based on incorporating fine-grained features from the reference
image directly into the latent feature maps of the main diffusion model,
alongside with a perceptual loss to further preserve the reference item's
details. We conduct extensive testing on both in-house and publicly available
datasets, and show that Diffuse to Choose is superior to existing zero-shot
diffusion inpainting methods as well as few-shot diffusion personalization
algorithms like DreamPaint.
Diffuse to Choose is a diffusion-based inpainting model that balances real-time performance with high-fidelity detail preservation and semantic accuracy.
diffusion models image-conditioned diffusion models fine-grained details latent feature maps perceptual loss zero-shot diffusion inpainting few-shot diffusion personalization DreamPaint