Q-Sched: Pushing the Boundaries of Few-Step Diffusion Models with
  Quantization-Aware Scheduling
Text-to-image diffusion models are computationally intensive, often requiring
dozens of forward passes through large transformer backbones. For instance,
Stable Diffusion XL generates high-quality images with 50 evaluations of a
2.6B-parameter model, an expensive process even for a single batch. Few-step
diffusion models reduce this cost to 2-8 denoising steps but still depend on
large, uncompressed U-Net or diffusion transformer backbones, which are often
too costly for full-precision inference without datacenter GPUs. These
requirements also limit existing post-training quantization methods that rely
on full-precision calibration. We introduce Q-Sched, a new paradigm for
post-training quantization that modifies the diffusion model scheduler rather
than model weights. By adjusting the few-step sampling trajectory, Q-Sched
achieves full-precision accuracy with a 4x reduction in model size. To learn
quantization-aware pre-conditioning coefficients, we propose the JAQ loss,
which combines text-image compatibility with an image quality metric for
fine-grained optimization. JAQ is reference-free and requires only a handful of
calibration prompts, avoiding full-precision inference during calibration.
Q-Sched delivers substantial gains: a 15.5% FID improvement over the FP16
4-step Latent Consistency Model and a 16.6% improvement over the FP16 8-step
Phased Consistency Model, showing that quantization and few-step distillation
are complementary for high-fidelity generation. A large-scale user study with
more than 80,000 annotations further confirms Q-Sched's effectiveness on both
FLUX.1[schnell] and SDXL-Turbo.
68b88f34d43cadaf7a688a87
Natalia Frumkin
68b88f34d43cadaf7a688a88
Diana Marculescu
Q-Sched: Pushing the Boundaries of Few-Step Diffusion Models with
  Quantization-Aware Scheduling
Text-to-image diffusion models are computationally intensive, often requiring
dozens of forward passes through large transformer backbones. For instance,
Stable Diffusion XL generates high-quality images with 50 evaluations of a
2.6B-parameter model, an expensive process even for a single batch. Few-step
diffusion models reduce this cost to 2-8 denoising steps but still depend on
large, uncompressed U-Net or diffusion transformer backbones, which are often
too costly for full-precision inference without datacenter GPUs. These
requirements also limit existing post-training quantization methods that rely
on full-precision calibration. We introduce Q-Sched, a new paradigm for
post-training quantization that modifies the diffusion model scheduler rather
than model weights. By adjusting the few-step sampling trajectory, Q-Sched
achieves full-precision accuracy with a 4x reduction in model size. To learn
quantization-aware pre-conditioning coefficients, we propose the JAQ loss,
which combines text-image compatibility with an image quality metric for
fine-grained optimization. JAQ is reference-free and requires only a handful of
calibration prompts, avoiding full-precision inference during calibration.
Q-Sched delivers substantial gains: a 15.5% FID improvement over the FP16
4-step Latent Consistency Model and a 16.6% improvement over the FP16 8-step
Phased Consistency Model, showing that quantization and few-step distillation
are complementary for high-fidelity generation. A large-scale user study with
more than 80,000 annotations further confirms Q-Sched's effectiveness on both
FLUX.1[schnell] and SDXL-Turbo.
Q-Sched, a novel post-training quantization method for diffusion models, reduces model size by 4x while maintaining full-precision accuracy and improving image quality metrics.
text-to-image diffusion models transformer backbones Stable Diffusion XL few-step diffusion models U-Net diffusion transformer backbones post-training quantization diffusion model scheduler quantization-aware pre-conditioning coefficients JAQ loss text-image compatibility image quality metric FID improvement Latent Consistency Model Phased Consistency Model FLUX.1 SDXL-Turbo