A Noise is Worth Diffusion Guidance
Diffusion models excel in generating high-quality images. However, current
diffusion models struggle to produce reliable images without guidance methods,
such as classifier-free guidance (CFG). Are guidance methods truly necessary?
Observing that noise obtained via diffusion inversion can reconstruct
high-quality images without guidance, we focus on the initial noise of the
denoising pipeline. By mapping Gaussian noise to `guidance-free noise', we
uncover that small low-magnitude low-frequency components significantly enhance
the denoising process, removing the need for guidance and thus improving both
inference throughput and memory. Expanding on this, we propose \ours, a novel
method that replaces guidance methods with a single refinement of the initial
noise. This refined noise enables high-quality image generation without
guidance, within the same diffusion pipeline. Our noise-refining model
leverages efficient noise-space learning, achieving rapid convergence and
strong performance with just 50K text-image pairs. We validate its
effectiveness across diverse metrics and analyze how refined noise can
eliminate the need for guidance. See our project page:
https://cvlab-kaist.github.io/NoiseRefine/.
6752a5285281c3cae4a81f18
Donghoon Ahn
6752a5285281c3cae4a81f19
Jiwon Kang
admin_assigned
2024-12-06T17:17:33.670Z
64bb4f6265b648b2df793ded
/avatars/fb384a47a4a2fa0c2c6319d0a8ade0b8.svg
Jiwon Kang
user
ji1kang
6752a5285281c3cae4a81f1a
Sanghyun Lee
6752a5285281c3cae4a81f1b
Jaewon Min
claimed_verified
2024-12-06T10:37:52.730Z
66012e9c9e1cf5eb41ee0c4c
/avatars/b07240cb86315b9e33d14677e02e4024.svg
Jaewon Min
user
Min-Jaewon
6752a5285281c3cae4a81f1c
Minjae Kim
6752a5285281c3cae4a81f1d
Wooseok Jang
6752a5285281c3cae4a81f1e
Hyoungwon Cho
admin_assigned
2024-12-06T17:16:07.961Z
6601282b569b30694e67b886
/avatars/bd6448ed822362b1470e91b749b920be.svg
Hyoungwon Cho
user
hyoungwoncho
6752a5285281c3cae4a81f1f
Sayak Paul
admin_assigned
2024-12-06T17:15:43.220Z
5f7fbd813e94f16a85448745
https://cdn-avatars.huggingface.co/v1/production/uploads/1649681653581-5f7fbd813e94f16a85448745.jpeg
Sayak Paul
user
sayakpaul
6752a5285281c3cae4a81f20
SeonHwa Kim
admin_assigned
2024-12-06T17:15:36.626Z
65ebb42e6b0697f087f1c443
/avatars/b19c15dbae55c08da9f472bde4073ada.svg
Seonhwa Kim
user
seonhwak
6752a5285281c3cae4a81f21
Eunju Cha
admin_assigned
2024-12-06T17:15:30.424Z
64d4d7e5badf1110f768cbe1
/avatars/3c67f3cbcb610dc906db59b6b73c16b8.svg
Eunju Cha
user
ejcha
6752a5285281c3cae4a81f22
Kyong Hwan Jin
admin_assigned
2024-12-06T17:15:13.536Z
6721a856329fc46a399cc988
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/nVNdvInvUw8_qvU8MY7XW.png
Kyong Hwan Jin
user
panakino
6752a5285281c3cae4a81f23
Seungryong Kim
admin_assigned
2024-12-06T17:15:06.238Z
65cf717450818a335a1d3021
/avatars/382a0e0f40f661cda1b2531e3e6ea2ee.svg
Seungryong Kim
user
seungryong
A Noise is Worth Diffusion Guidance
Diffusion models excel in generating high-quality images. However, current
diffusion models struggle to produce reliable images without guidance methods,
such as classifier-free guidance (CFG). Are guidance methods truly necessary?
Observing that noise obtained via diffusion inversion can reconstruct
high-quality images without guidance, we focus on the initial noise of the
denoising pipeline. By mapping Gaussian noise to `guidance-free noise', we
uncover that small low-magnitude low-frequency components significantly enhance
the denoising process, removing the need for guidance and thus improving both
inference throughput and memory. Expanding on this, we propose \ours, a novel
method that replaces guidance methods with a single refinement of the initial
noise. This refined noise enables high-quality image generation without
guidance, within the same diffusion pipeline. Our noise-refining model
leverages efficient noise-space learning, achieving rapid convergence and
strong performance with just 50K text-image pairs. We validate its
effectiveness across diverse metrics and analyze how refined noise can
eliminate the need for guidance. See our project page:
https://cvlab-kaist.github.io/NoiseRefine/.
A noise-refining method is introduced to eliminate the need for guidance in diffusion models, enhancing image generation speed and memory efficiency.
diffusion models classifier-free guidance diffusion inversion denoising pipeline Gaussian noise efficient noise-space learning