Weak-to-Strong Diffusion with Reflection
The goal of diffusion generative models is to align the learned distribution
with the real data distribution through gradient score matching. However,
inherent limitations in training data quality, modeling strategies, and
architectural design lead to inevitable gap between generated outputs and real
data. To reduce this gap, we propose Weak-to-Strong Diffusion (W2SD), a novel
framework that utilizes the estimated difference between existing weak and
strong models (i.e., weak-to-strong difference) to approximate the gap between
an ideal model and a strong model. By employing a reflective operation that
alternates between denoising and inversion with weak-to-strong difference, we
theoretically understand that W2SD steers latent variables along sampling
trajectories toward regions of the real data distribution. W2SD is highly
flexible and broadly applicable, enabling diverse improvements through the
strategic selection of weak-to-strong model pairs (e.g., DreamShaper vs. SD1.5,
good experts vs. bad experts in MoE). Extensive experiments demonstrate that
W2SD significantly improves human preference, aesthetic quality, and prompt
adherence, achieving SOTA performance across various modalities (e.g., image,
video), architectures (e.g., UNet-based, DiT-based, MoE), and benchmarks. For
example, Juggernaut-XL with W2SD can improve with the HPSv2 winning rate up to
90% over the original results. Moreover, the performance gains achieved by W2SD
markedly outweigh its additional computational overhead, while the cumulative
improvements from different weak-to-strong difference further solidify its
practical utility and deployability.
67a5f635c20315f5e3f16f62
Lichen Bai
claimed_verified
2025-02-07T12:56:27.462Z
66348bf4e1555067669870fa
/avatars/8b8bbc7dff7d9a0a02b0960084bc95ab.svg
白立忱
user
Indulge-Bai
67a5f635c20315f5e3f16f63
Masashi Sugiyama
67a5f635c20315f5e3f16f64
Zeke Xie
claimed_verified
2025-03-16T21:20:47.773Z
67a5fb17baac478d94c59fec
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/gMtdNiD8ztyAjB_weexcK.png
Z.X.
user
xLeaF
Weak-to-Strong Diffusion with Reflection
The goal of diffusion generative models is to align the learned distribution
with the real data distribution through gradient score matching. However,
inherent limitations in training data quality, modeling strategies, and
architectural design lead to inevitable gap between generated outputs and real
data. To reduce this gap, we propose Weak-to-Strong Diffusion (W2SD), a novel
framework that utilizes the estimated difference between existing weak and
strong models (i.e., weak-to-strong difference) to approximate the gap between
an ideal model and a strong model. By employing a reflective operation that
alternates between denoising and inversion with weak-to-strong difference, we
theoretically understand that W2SD steers latent variables along sampling
trajectories toward regions of the real data distribution. W2SD is highly
flexible and broadly applicable, enabling diverse improvements through the
strategic selection of weak-to-strong model pairs (e.g., DreamShaper vs. SD1.5,
good experts vs. bad experts in MoE). Extensive experiments demonstrate that
W2SD significantly improves human preference, aesthetic quality, and prompt
adherence, achieving SOTA performance across various modalities (e.g., image,
video), architectures (e.g., UNet-based, DiT-based, MoE), and benchmarks. For
example, Juggernaut-XL with W2SD can improve with the HPSv2 winning rate up to
90% over the original results. Moreover, the performance gains achieved by W2SD
markedly outweigh its additional computational overhead, while the cumulative
improvements from different weak-to-strong difference further solidify its
practical utility and deployability.
The Weak-to-Strong Diffusion (W2SD) framework enhances the alignment of learned distributions with real data distributions by leveraging differences between weak and strong models, leading to significant improvements in human preference, aesthetic quality, and prompt adherence across multiple modalities and architectures.
diffusion generative models gradient score matching weak-to-strong diffusion W2SD denoising inversion latent variables ideal model sampling trajectories real data distribution human preference aesthetic quality prompt adherence SOTA performance Juggernaut-XL HPSv2 UNet-based DiT-based MoE