Proper Hölder–Kullback Dirichlet Diffusion: A Framework for High-Dimensional Generative Modeling
Wanpeng Zhang, Yuhao Fang, Xihang Qiu, Jiarong Cheng, Jialong Hong, Bin Zhai, Qing Zhou, Yao Lu, Ye Zhang, Chun Li
Diffusion-based generative models have long depended on Gaussian priors, with little exploration of alternative distributions. To fill this gap, we introduce a scaled and shifted Dirichlet framework that uses time-varying multiplicative transformations to define both forward and reverse diffusion processes. Moving beyond conventional reweighted evidence lower bounds (ELBO) or Kullback–Leibler upper bounds (KLUB), we propose two novel divergence measures: the Proper Hölder Divergence (PHD) and the Proper Hölder–Kullback (PHK) divergence, the latter designed to restore symmetry missing in existing formulations. When optimizing our Dirichlet diffusion model with PHK, we achieve a Fréchet Inception Distance (FID) of 2.81 on unconditional CIFAR-10. Comprehensive experiments on natural-image datasets validate the distinct generative strengths of Dirichlet diffusion and confirm PHK’s effectiveness in model training. These contributions expand the diffusion-model family with principled non-Gaussian processes and robust optimization tools, offering new avenues for versatile, high-fidelity generative modeling.