Fast-dLLM v2: Efficient Block-Diffusion LLM
Autoregressive (AR) large language models (LLMs) have achieved remarkable
performance across a wide range of natural language tasks, yet their inherent
sequential decoding limits inference efficiency. In this work, we propose
Fast-dLLM v2, a carefully designed block diffusion language model (dLLM) that
efficiently adapts pretrained AR models into dLLMs for parallel text
generation, requiring only approximately 1B tokens of fine-tuning. This
represents a 500x reduction in training data compared to full-attention
diffusion LLMs such as Dream (580B tokens), while preserving the original
model's performance. Our approach introduces a novel training recipe that
combines a block diffusion mechanism with a complementary attention mask,
enabling blockwise bidirectional context modeling without sacrificing AR
training objectives. To further accelerate decoding, we design a hierarchical
caching mechanism: a block-level cache that stores historical context
representations across blocks, and a sub-block cache that enables efficient
parallel generation within partially decoded blocks. Coupled with our parallel
decoding pipeline, Fast-dLLM v2 achieves up to 2.5x speedup over standard AR
decoding without compromising generation quality. Extensive experiments across
diverse benchmarks demonstrate that Fast-dLLM v2 matches or surpasses AR
baselines in accuracy, while delivering state-of-the-art efficiency among dLLMs
- marking a significant step toward the practical deployment of fast and
accurate LLMs. Code and model will be publicly released.
68e1e6d873e20ab577841e18
Chengyue Wu
68e1e6d873e20ab577841e19
Hao Zhang
68e1e6d873e20ab577841e1a
Shuchen Xue
68e1e6d873e20ab577841e1b
Shizhe Diao
68e1e6d873e20ab577841e1c
Yonggan Fu
68e1e6d873e20ab577841e1d
Zhijian Liu
claimed_verified
2025-10-08T08:03:25.561Z
650dac79b959b0e1d41d7378
https://cdn-avatars.huggingface.co/v1/production/uploads/650dac79b959b0e1d41d7378/mzbN0MFk3k8b94FQ40I7L.jpeg
Zhijian Liu
user
zhijianliu
68e1e6d873e20ab577841e1e
Pavlo Molchanov
68e1e6d873e20ab577841e1f
Ping Luo
68e1e6d873e20ab577841e20
Song Han
68e1e6d873e20ab577841e21
Enze Xie
Fast-dLLM v2: Efficient Block-Diffusion LLM
Autoregressive (AR) large language models (LLMs) have achieved remarkable
performance across a wide range of natural language tasks, yet their inherent
sequential decoding limits inference efficiency. In this work, we propose
Fast-dLLM v2, a carefully designed block diffusion language model (dLLM) that
efficiently adapts pretrained AR models into dLLMs for parallel text
generation, requiring only approximately 1B tokens of fine-tuning. This
represents a 500x reduction in training data compared to full-attention
diffusion LLMs such as Dream (580B tokens), while preserving the original
model's performance. Our approach introduces a novel training recipe that
combines a block diffusion mechanism with a complementary attention mask,
enabling blockwise bidirectional context modeling without sacrificing AR
training objectives. To further accelerate decoding, we design a hierarchical
caching mechanism: a block-level cache that stores historical context
representations across blocks, and a sub-block cache that enables efficient
parallel generation within partially decoded blocks. Coupled with our parallel
decoding pipeline, Fast-dLLM v2 achieves up to 2.5x speedup over standard AR
decoding without compromising generation quality. Extensive experiments across
diverse benchmarks demonstrate that Fast-dLLM v2 matches or surpasses AR
baselines in accuracy, while delivering state-of-the-art efficiency among dLLMs
- marking a significant step toward the practical deployment of fast and
accurate LLMs. Code and model will be publicly released.
Fast-dLLM v2, a block diffusion language model, efficiently converts pretrained autoregressive models for parallel text generation, achieving significant speedup without compromising accuracy.
autoregressive models large language models diffusion language models block diffusion mechanism attention mask blockwise bidirectional context modeling hierarchical caching block-level cache sub-block cache parallel decoding pipeline