Diffusion Language Models Know the Answer Before Decoding
Diffusion language models (DLMs) have recently emerged as an alternative to
autoregressive approaches, offering parallel sequence generation and flexible
token orders. However, their inference remains slower than that of
autoregressive models, primarily due to the cost of bidirectional attention and
the large number of refinement steps required for high quality outputs. In this
work, we highlight and leverage an overlooked property of DLMs early answer
convergence: in many cases, the correct answer can be internally identified by
half steps before the final decoding step, both under semi-autoregressive and
random remasking schedules. For example, on GSM8K and MMLU, up to 97% and 99%
of instances, respectively, can be decoded correctly using only half of the
refinement steps. Building on this observation, we introduce Prophet, a
training-free fast decoding paradigm that enables early commit decoding.
Specifically, Prophet dynamically decides whether to continue refinement or to
go "all-in" (i.e., decode all remaining tokens in one step), using the
confidence gap between the top-2 prediction candidates as the criterion. It
integrates seamlessly into existing DLM implementations, incurs negligible
overhead, and requires no additional training. Empirical evaluations of
LLaDA-8B and Dream-7B across multiple tasks show that Prophet reduces the
number of decoding steps by up to 3.4x while preserving high generation
quality. These results recast DLM decoding as a problem of when to stop
sampling, and demonstrate that early decode convergence provides a simple yet
powerful mechanism for accelerating DLM inference, complementary to existing
speedup techniques. Our code is publicly available at
https://github.com/pixeli99/Prophet.
68afd84d245176306494ccad
Pengxiang Li
admin_assigned
2025-08-28T18:58:55.979Z
64245f2c089d5fae56b4549a
https://cdn-avatars.huggingface.co/v1/production/uploads/64245f2c089d5fae56b4549a/qUHFsL9Svwyj5BKpfMtaY.jpeg
Pengxiang Li
user
pengxiang
68afd84d245176306494ccae
Yefan Zhou
68afd84d245176306494ccaf
Dilxat Muhtar
admin_assigned
2025-08-28T18:59:02.063Z
631551858be84e1ab41478f4
/avatars/c464f009ec0b7ec8e144ab8fbd07de43.svg
Dilxat Muhtar
user
PumpkinCat
68afd84d245176306494ccb0
Lu Yin
68afd84d245176306494ccb1
Shilin Yan
68afd84d245176306494ccb2
Li Shen
68afd84d245176306494ccb3
Yi Liang
68afd84d245176306494ccb4
Soroush Vosoughi
admin_assigned
2025-08-28T19:00:53.501Z
648169d3cb64a2b85a4606e6
https://cdn-avatars.huggingface.co/v1/production/uploads/648169d3cb64a2b85a4606e6/6nEeE3TWvO9O5dCJTOG0C.jpeg
Soroush Vosoughi
user
soroushv
68afd84d245176306494ccb5
Shiwei Liu
Diffusion Language Models Know the Answer Before Decoding
Diffusion language models (DLMs) have recently emerged as an alternative to
autoregressive approaches, offering parallel sequence generation and flexible
token orders. However, their inference remains slower than that of
autoregressive models, primarily due to the cost of bidirectional attention and
the large number of refinement steps required for high quality outputs. In this
work, we highlight and leverage an overlooked property of DLMs early answer
convergence: in many cases, the correct answer can be internally identified by
half steps before the final decoding step, both under semi-autoregressive and
random remasking schedules. For example, on GSM8K and MMLU, up to 97% and 99%
of instances, respectively, can be decoded correctly using only half of the
refinement steps. Building on this observation, we introduce Prophet, a
training-free fast decoding paradigm that enables early commit decoding.
Specifically, Prophet dynamically decides whether to continue refinement or to
go "all-in" (i.e., decode all remaining tokens in one step), using the
confidence gap between the top-2 prediction candidates as the criterion. It
integrates seamlessly into existing DLM implementations, incurs negligible
overhead, and requires no additional training. Empirical evaluations of
LLaDA-8B and Dream-7B across multiple tasks show that Prophet reduces the
number of decoding steps by up to 3.4x while preserving high generation
quality. These results recast DLM decoding as a problem of when to stop
sampling, and demonstrate that early decode convergence provides a simple yet
powerful mechanism for accelerating DLM inference, complementary to existing
speedup techniques. Our code is publicly available at
https://github.com/pixeli99/Prophet.
Prophet, a training-free fast decoding paradigm for diffusion language models, reduces inference time by leveraging early answer convergence without sacrificing quality.
diffusion language models DLMs autoregressive approaches parallel sequence generation bidirectional attention refinement steps early answer convergence semi-autoregressive random remasking schedules Prophet confidence gap top-2 prediction candidates early commit decoding LLaDA-8B Dream-7B decoding steps generation quality