Planned Diffusion
A central challenge in large language model inference is the trade-off
between generation speed and output quality. Autoregressive models produce
high-quality text but generate tokens sequentially. Diffusion models can
generate tokens in parallel but often need many iterations to match the same
quality. We propose planned diffusion, a hybrid method that combines the
strengths of both paradigms. Planned diffusion works in two stages: first, the
model creates a short autoregressive plan that breaks the output into smaller,
independent spans. Second, the model generates these spans simultaneously using
diffusion. This approach expands the speed-quality Pareto frontier and provides
a practical path to faster, high-quality text generation. On AlpacaEval, a
suite of 805 instruction-following prompts, planned diffusion achieves
Pareto-optimal trade-off between quality and latency, achieving 1.27x to 1.81x
speedup over autoregressive generation with only 0.87\% to 5.4\% drop in win
rate, respectively. Our sensitivity analysis shows that the planning mechanism
of planned diffusion is minimal and reliable, and simple runtime knobs exist to
provide flexible control of the quality-latency trade-off.
68f821b07669bcaeecce0b6f
Daniel Israel
claimed_verified
2025-10-23T15:00:33.524Z
630139f1f6bea7dd15bdaf4e
/avatars/263536f6160f8c522d2a76ca2c4e4cc0.svg
Daniel Israel
user
danielmisrael
68f821b07669bcaeecce0b70
Tian Jin
68f821b07669bcaeecce0b71
Ellie Cheng
68f821b07669bcaeecce0b72
Guy Van den Broeck
68f821b07669bcaeecce0b73
Aditya Grover
68f821b07669bcaeecce0b74
Suvinay Subramanian
68f821b07669bcaeecce0b75
Michael Carbin
Planned Diffusion
A central challenge in large language model inference is the trade-off
between generation speed and output quality. Autoregressive models produce
high-quality text but generate tokens sequentially. Diffusion models can
generate tokens in parallel but often need many iterations to match the same
quality. We propose planned diffusion, a hybrid method that combines the
strengths of both paradigms. Planned diffusion works in two stages: first, the
model creates a short autoregressive plan that breaks the output into smaller,
independent spans. Second, the model generates these spans simultaneously using
diffusion. This approach expands the speed-quality Pareto frontier and provides
a practical path to faster, high-quality text generation. On AlpacaEval, a
suite of 805 instruction-following prompts, planned diffusion achieves
Pareto-optimal trade-off between quality and latency, achieving 1.27x to 1.81x
speedup over autoregressive generation with only 0.87\% to 5.4\% drop in win
rate, respectively. Our sensitivity analysis shows that the planning mechanism
of planned diffusion is minimal and reliable, and simple runtime knobs exist to
provide flexible control of the quality-latency trade-off.
Planned diffusion combines autoregressive and diffusion models to achieve faster text generation with minimal quality loss.
autoregressive models diffusion models planned diffusion Pareto frontier AlpacaEval win rate sensitivity analysis