Adaptive Classifier-Free Guidance via Dynamic Low-Confidence Masking
Classifier-Free Guidance (CFG) significantly enhances controllability in
generative models by interpolating conditional and unconditional predictions.
However, standard CFG often employs a static unconditional input, which can be
suboptimal for iterative generation processes where model uncertainty varies
dynamically. We introduce Adaptive Classifier-Free Guidance (A-CFG), a novel
method that tailors the unconditional input by leveraging the model's
instantaneous predictive confidence. At each step of an iterative (masked)
diffusion language model, A-CFG identifies tokens in the currently generated
sequence for which the model exhibits low confidence. These tokens are
temporarily re-masked to create a dynamic, localized unconditional input. This
focuses CFG's corrective influence precisely on areas of ambiguity, leading to
more effective guidance. We integrate A-CFG into a state-of-the-art masked
diffusion language model and demonstrate its efficacy. Experiments on diverse
language generation benchmarks show that A-CFG yields substantial improvements
over standard CFG, achieving, for instance, a 3.9 point gain on GPQA. Our work
highlights the benefit of dynamically adapting guidance mechanisms to model
uncertainty in iterative generation.
683919611186f2cbf3ed2267
Pengxiang Li
claimed_verified
2025-09-26T12:26:14.211Z
64245f2c089d5fae56b4549a
https://cdn-avatars.huggingface.co/v1/production/uploads/64245f2c089d5fae56b4549a/qUHFsL9Svwyj5BKpfMtaY.jpeg
Pengxiang Li
user
pengxiang
683919611186f2cbf3ed2268
Shilin Yan
683919611186f2cbf3ed2269
Joey Tsai
683919611186f2cbf3ed226a
Renrui Zhang
683919611186f2cbf3ed226b
Ruichuan An
683919611186f2cbf3ed226c
Ziyu Guo
683919611186f2cbf3ed226d
Xiaowei Gao
Adaptive Classifier-Free Guidance via Dynamic Low-Confidence Masking
Classifier-Free Guidance (CFG) significantly enhances controllability in
generative models by interpolating conditional and unconditional predictions.
However, standard CFG often employs a static unconditional input, which can be
suboptimal for iterative generation processes where model uncertainty varies
dynamically. We introduce Adaptive Classifier-Free Guidance (A-CFG), a novel
method that tailors the unconditional input by leveraging the model's
instantaneous predictive confidence. At each step of an iterative (masked)
diffusion language model, A-CFG identifies tokens in the currently generated
sequence for which the model exhibits low confidence. These tokens are
temporarily re-masked to create a dynamic, localized unconditional input. This
focuses CFG's corrective influence precisely on areas of ambiguity, leading to
more effective guidance. We integrate A-CFG into a state-of-the-art masked
diffusion language model and demonstrate its efficacy. Experiments on diverse
language generation benchmarks show that A-CFG yields substantial improvements
over standard CFG, achieving, for instance, a 3.9 point gain on GPQA. Our work
highlights the benefit of dynamically adapting guidance mechanisms to model
uncertainty in iterative generation.
Adaptive Classifier-Free Guidance (A-CFG) dynamically adjusts the guidance in masked diffusion language models by focusing on areas of low model confidence, leading to significant improvements in language generation performance.
Classifier-Free Guidance (CFG) Adaptive Classifier-Free Guidance (A-CFG) masked diffusion language model predictive confidence GPQA