d^2Cache: Accelerating Diffusion-Based LLMs via Dual Adaptive Caching
Diffusion-based large language models (dLLMs), despite their promising
performance, still suffer from inferior inference efficiency. This is because
dLLMs rely on bidirectional attention and cannot directly benefit from the
standard key-value (KV) cache as autoregressive models (ARMs) do. To tackle
this issue, we introduce Dual aDaptive Cache (d^2Cache), which is a
training-free approximate KV cache framework for accelerating dLLM inference.
d^2Cache features a two-stage fine-grained selection strategy to identify
tokens and adaptively update their KV states at each decoding step, while
caching the KV states of the remaining tokens for reuse. Furthermore,
d^2Cache naturally offers a more reliable decoding alternative, which can
enable quasi left-to-right generation and mitigate premature overconfidence in
tokens at the end of the sequence. Extensive experimental results on two
representative dLLMs (\ie, LLaDA and Dream) demonstrate that d^2Cache not
only achieves substantial inference speedups, but also yields consistent
improvements in generation quality. The code is available at
https://github.com/Kamichanw/d2Cache.
68db35a9d2bf1f4b15ec727b
Yuchu Jiang
68db35a9d2bf1f4b15ec727c
Yue Cai
68db35a9d2bf1f4b15ec727d
Xiangzhong Luo
68db35a9d2bf1f4b15ec727e
Jiale Fu
68db35a9d2bf1f4b15ec727f
Jiarui Wang
68db35a9d2bf1f4b15ec7280
Chonghan Liu
68db35a9d2bf1f4b15ec7281
Xu Yang
d^2Cache: Accelerating Diffusion-Based LLMs via Dual Adaptive Caching
Diffusion-based large language models (dLLMs), despite their promising
performance, still suffer from inferior inference efficiency. This is because
dLLMs rely on bidirectional attention and cannot directly benefit from the
standard key-value (KV) cache as autoregressive models (ARMs) do. To tackle
this issue, we introduce Dual aDaptive Cache (d^2Cache), which is a
training-free approximate KV cache framework for accelerating dLLM inference.
d^2Cache features a two-stage fine-grained selection strategy to identify
tokens and adaptively update their KV states at each decoding step, while
caching the KV states of the remaining tokens for reuse. Furthermore,
d^2Cache naturally offers a more reliable decoding alternative, which can
enable quasi left-to-right generation and mitigate premature overconfidence in
tokens at the end of the sequence. Extensive experimental results on two
representative dLLMs (\ie, LLaDA and Dream) demonstrate that d^2Cache not
only achieves substantial inference speedups, but also yields consistent
improvements in generation quality. The code is available at
https://github.com/Kamichanw/d2Cache.
Dual aDaptive Cache (d²Cache) accelerates diffusion-based large language model inference by selectively updating key-value states and enabling quasi left-to-right generation, improving both speed and quality.
diffusion-based large language models dLLMs bidirectional attention key-value cache autoregressive models Dual aDaptive Cache d²Cache two-stage fine-grained selection quasi left-to-right generation LLaDA Dream