SVNR: Spatially-variant Noise Removal with Denoising Diffusion
Denoising diffusion models have recently shown impressive results in
generative tasks. By learning powerful priors from huge collections of training
images, such models are able to gradually modify complete noise to a clean
natural image via a sequence of small denoising steps, seemingly making them
well-suited for single image denoising. However, effectively applying denoising
diffusion models to removal of realistic noise is more challenging than it may
seem, since their formulation is based on additive white Gaussian noise, unlike
noise in real-world images. In this work, we present SVNR, a novel formulation
of denoising diffusion that assumes a more realistic, spatially-variant noise
model. SVNR enables using the noisy input image as the starting point for the
denoising diffusion process, in addition to conditioning the process on it. To
this end, we adapt the diffusion process to allow each pixel to have its own
time embedding, and propose training and inference schemes that support
spatially-varying time maps. Our formulation also accounts for the correlation
that exists between the condition image and the samples along the modified
diffusion process. In our experiments we demonstrate the advantages of our
approach over a strong diffusion model baseline, as well as over a
state-of-the-art single image denoising method.
649cf1dde89a765015a0e68b
Naama Pearl
claimed_verified
2023-07-12T07:30:00.872Z
64ad7c56b7d86b40fd84f39b
/avatars/6ddd82f63bc9c91996e9aa41561d63ba.svg
Naama Pearl
user
naamapearl
649cf1dde89a765015a0e68c
Yaron Brodsky
649cf1dde89a765015a0e68d
Dana Berman
649cf1dde89a765015a0e68e
Assaf Zomet
649cf1dde89a765015a0e68f
Alex Rav Acha
649cf1dde89a765015a0e690
Daniel Cohen-Or
649cf1dde89a765015a0e691
Dani Lischinski
SVNR: Spatially-variant Noise Removal with Denoising Diffusion
Denoising diffusion models have recently shown impressive results in
generative tasks. By learning powerful priors from huge collections of training
images, such models are able to gradually modify complete noise to a clean
natural image via a sequence of small denoising steps, seemingly making them
well-suited for single image denoising. However, effectively applying denoising
diffusion models to removal of realistic noise is more challenging than it may
seem, since their formulation is based on additive white Gaussian noise, unlike
noise in real-world images. In this work, we present SVNR, a novel formulation
of denoising diffusion that assumes a more realistic, spatially-variant noise
model. SVNR enables using the noisy input image as the starting point for the
denoising diffusion process, in addition to conditioning the process on it. To
this end, we adapt the diffusion process to allow each pixel to have its own
time embedding, and propose training and inference schemes that support
spatially-varying time maps. Our formulation also accounts for the correlation
that exists between the condition image and the samples along the modified
diffusion process. In our experiments we demonstrate the advantages of our
approach over a strong diffusion model baseline, as well as over a
state-of-the-art single image denoising method.
SVNR, a novel denoising diffusion model, effectively handles realistic, spatially-variant noise by adapting the diffusion process with pixel-wise time embeddings and conditioning.
denoising diffusion models SVNR spatially-variant noise model time embedding diffusion process denoising steps