Time Is a Feature: Exploiting Temporal Dynamics in Diffusion Language
  Models
Diffusion large language models (dLLMs) generate text through iterative
denoising, yet current decoding strategies discard rich intermediate
predictions in favor of the final output. Our work here reveals a critical
phenomenon, temporal oscillation, where correct answers often emerge in the
middle process, but are overwritten in later denoising steps. To address this
issue, we introduce two complementary methods that exploit temporal
consistency: 1) Temporal Self-Consistency Voting, a training-free, test-time
decoding strategy that aggregates predictions across denoising steps to select
the most consistent output; and 2) a post-training method termed Temporal
Consistency Reinforcement, which uses Temporal Semantic Entropy (TSE), a
measure of semantic stability across intermediate predictions, as a reward
signal to encourage stable generations. Empirical results across multiple
benchmarks demonstrate the effectiveness of our approach. Using the negative
TSE reward alone, we observe a remarkable average improvement of 24.7% on the
Countdown dataset over an existing dLLM. Combined with the accuracy reward, we
achieve absolute gains of 2.0% on GSM8K, 4.3% on MATH500, 6.6% on SVAMP, and
25.3% on Countdown, respectively. Our findings underscore the untapped
potential of temporal dynamics in dLLMs and offer two simple yet effective
tools to harness them.
689befdbfab6fdd2e52ac80c
Wen Wang
claimed_verified
2025-08-14T13:38:56.121Z
63f089456309c84d5f47f951
/avatars/04b926a7f2ad091ee00fef0c59903492.svg
Wen Wang
user
wwen1997
689befdbfab6fdd2e52ac80d
Bozhen Fang
689befdbfab6fdd2e52ac80e
Chenchen Jing
689befdbfab6fdd2e52ac80f
Yongliang Shen
claimed_verified
2025-08-13T07:13:02.765Z
5e1058e9fcf41d740b69966d
/avatars/ce74839ba871f2b54313a670a233ba82.svg
Yongliang Shen
user
tricktreat
689befdbfab6fdd2e52ac810
Yangyi Shen
689befdbfab6fdd2e52ac811
Qiuyu Wang
claimed_verified
2025-08-13T07:13:07.409Z
64981bea09cea550852652af
/avatars/df528e9008972c8e5ae4d278e617476c.svg
Qiuyu Wang
user
qiuyuu
689befdbfab6fdd2e52ac812
Hao Ouyang
689befdbfab6fdd2e52ac813
Hao Chen
689befdbfab6fdd2e52ac814
Chunhua Shen
Time Is a Feature: Exploiting Temporal Dynamics in Diffusion Language
  Models
Diffusion large language models (dLLMs) generate text through iterative
denoising, yet current decoding strategies discard rich intermediate
predictions in favor of the final output. Our work here reveals a critical
phenomenon, temporal oscillation, where correct answers often emerge in the
middle process, but are overwritten in later denoising steps. To address this
issue, we introduce two complementary methods that exploit temporal
consistency: 1) Temporal Self-Consistency Voting, a training-free, test-time
decoding strategy that aggregates predictions across denoising steps to select
the most consistent output; and 2) a post-training method termed Temporal
Consistency Reinforcement, which uses Temporal Semantic Entropy (TSE), a
measure of semantic stability across intermediate predictions, as a reward
signal to encourage stable generations. Empirical results across multiple
benchmarks demonstrate the effectiveness of our approach. Using the negative
TSE reward alone, we observe a remarkable average improvement of 24.7% on the
Countdown dataset over an existing dLLM. Combined with the accuracy reward, we
achieve absolute gains of 2.0% on GSM8K, 4.3% on MATH500, 6.6% on SVAMP, and
25.3% on Countdown, respectively. Our findings underscore the untapped
potential of temporal dynamics in dLLMs and offer two simple yet effective
tools to harness them.
Two methods, Temporal Self-Consistency Voting and Temporal Consistency Reinforcement, improve diffusion large language models by leveraging temporal consistency in intermediate predictions.
diffusion large language models denoising temporal oscillation Temporal Self-Consistency Voting Temporal Consistency Reinforcement Temporal Semantic Entropy semantic stability Countdown dataset GSM8K MATH500 SVAMP