Direct Discriminative Optimization: Your Likelihood-Based Visual
  Generative Model is Secretly a GAN Discriminator
While likelihood-based generative models, particularly diffusion and
autoregressive models, have achieved remarkable fidelity in visual generation,
the maximum likelihood estimation (MLE) objective inherently suffers from a
mode-covering tendency that limits the generation quality under limited model
capacity. In this work, we propose Direct Discriminative Optimization (DDO) as
a unified framework that bridges likelihood-based generative training and the
GAN objective to bypass this fundamental constraint. Our key insight is to
parameterize a discriminator implicitly using the likelihood ratio between a
learnable target model and a fixed reference model, drawing parallels with the
philosophy of Direct Preference Optimization (DPO). Unlike GANs, this
parameterization eliminates the need for joint training of generator and
discriminator networks, allowing for direct, efficient, and effective
finetuning of a well-trained model to its full potential beyond the limits of
MLE. DDO can be performed iteratively in a self-play manner for progressive
model refinement, with each round requiring less than 1% of pretraining epochs.
Our experiments demonstrate the effectiveness of DDO by significantly advancing
the previous SOTA diffusion model EDM, reducing FID scores from 1.79/1.58 to
new records of 1.30/0.97 on CIFAR-10/ImageNet-64 datasets, and by consistently
improving both guidance-free and CFG-enhanced FIDs of visual autoregressive
models on ImageNet 256times256.
67c6d1c35e896ed915374027
Kaiwen Zheng
claimed_verified
2025-03-04T10:17:24.142Z
652bf7edc3cba555d5673c6e
/avatars/78f6416c30203b30671f8423f061c657.svg
Kaiwen Zheng
user
worstcoder
67c6d1c35e896ed915374028
Yongxin Chen
admin_assigned
2025-03-04T11:16:59.170Z
66f4cf1a03b5ba8a7f1f6522
/avatars/2768d6e37d3f280194cfb8ed274f6015.svg
Yongxin Chen
user
Ema11
67c6d1c35e896ed915374029
Huayu Chen
admin_assigned
2025-03-04T11:17:06.080Z
6630f87ee53fcb71c3887df0
/avatars/50191a3d45bebf90cf08df09477e95db.svg
HuayuChen
user
HuayuChen
67c6d1c35e896ed91537402a
Guande He
admin_assigned
2025-03-04T11:17:20.266Z
67492ee82ad3cfc108a41bbb
/avatars/7ad03e55a8791c62f1271a5c9bf8cc60.svg
Guande He
user
gdhe17
67c6d1c35e896ed91537402b
Ming-Yu Liu
admin_assigned
2025-03-04T11:17:27.270Z
62f049afdf4b93aad5c7f2d6
/avatars/e272e58ad996733d7098e50248e5b57e.svg
Ming-Yu Liu
user
mingyuliutw
67c6d1c35e896ed91537402c
Jun Zhu
67c6d1c35e896ed91537402d
Qinsheng Zhang
admin_assigned
2025-03-04T11:17:33.763Z
6732d5dea24987c43bfbafd8
/avatars/1581373b9de5069975716932fceb976b.svg
Qinsheng Zhang
user
qsh-zh
Direct Discriminative Optimization: Your Likelihood-Based Visual
  Generative Model is Secretly a GAN Discriminator
While likelihood-based generative models, particularly diffusion and
autoregressive models, have achieved remarkable fidelity in visual generation,
the maximum likelihood estimation (MLE) objective inherently suffers from a
mode-covering tendency that limits the generation quality under limited model
capacity. In this work, we propose Direct Discriminative Optimization (DDO) as
a unified framework that bridges likelihood-based generative training and the
GAN objective to bypass this fundamental constraint. Our key insight is to
parameterize a discriminator implicitly using the likelihood ratio between a
learnable target model and a fixed reference model, drawing parallels with the
philosophy of Direct Preference Optimization (DPO). Unlike GANs, this
parameterization eliminates the need for joint training of generator and
discriminator networks, allowing for direct, efficient, and effective
finetuning of a well-trained model to its full potential beyond the limits of
MLE. DDO can be performed iteratively in a self-play manner for progressive
model refinement, with each round requiring less than 1% of pretraining epochs.
Our experiments demonstrate the effectiveness of DDO by significantly advancing
the previous SOTA diffusion model EDM, reducing FID scores from 1.79/1.58 to
new records of 1.30/0.97 on CIFAR-10/ImageNet-64 datasets, and by consistently
improving both guidance-free and CFG-enhanced FIDs of visual autoregressive
models on ImageNet 256times256.
A unified framework called Direct Discriminative Optimization (DDO) enhances generative models by combining likelihood-based and GAN objectives, improving model fidelity and efficiency without joint training.
likelihood-based generative models diffusion models autoregressive models maximum likelihood estimation (MLE) Direct Discriminative Optimization (DDO) discriminator likelihood ratio Direct Preference Optimization (DPO) generator SOTA diffusion model EDM FID scores guidance-free CFG-enhanced ImageNet-64 ImageNet 256$\times$256