Self-conditioned Image Generation via Generating Representations
This paper presents Representation-Conditioned image
Generation (RCG), a simple yet effective image generation framework
which sets a new benchmark in class-unconditional image generation. RCG does
not condition on any human annotations. Instead, it conditions on a
self-supervised representation distribution which is mapped from the image
distribution using a pre-trained encoder. During generation, RCG samples from
such representation distribution using a representation diffusion model (RDM),
and employs a pixel generator to craft image pixels conditioned on the sampled
representation. Such a design provides substantial guidance during the
generative process, resulting in high-quality image generation. Tested on
ImageNet 256times256, RCG achieves a Frechet Inception Distance (FID) of
3.31 and an Inception Score (IS) of 253.4. These results not only significantly
improve the state-of-the-art of class-unconditional image generation but also
rival the current leading methods in class-conditional image generation,
bridging the long-standing performance gap between these two tasks. Code is
available at https://github.com/LTH14/rcg.
657132653e68d88fd61f1fb2
Tianhong Li
admin_assigned
2023-12-07T12:18:39.736Z
65244352e0823fbfef8c6475
https://cdn-avatars.huggingface.co/v1/production/uploads/65244352e0823fbfef8c6475/d6YipDIYHdhP_B6RJACNc.jpeg
Tianhong Li
user
LTH14
657132653e68d88fd61f1fb3
Dina Katabi
657132653e68d88fd61f1fb4
Kaiming He
Self-conditioned Image Generation via Generating Representations
This paper presents Representation-Conditioned image
Generation (RCG), a simple yet effective image generation framework
which sets a new benchmark in class-unconditional image generation. RCG does
not condition on any human annotations. Instead, it conditions on a
self-supervised representation distribution which is mapped from the image
distribution using a pre-trained encoder. During generation, RCG samples from
such representation distribution using a representation diffusion model (RDM),
and employs a pixel generator to craft image pixels conditioned on the sampled
representation. Such a design provides substantial guidance during the
generative process, resulting in high-quality image generation. Tested on
ImageNet 256times256, RCG achieves a Frechet Inception Distance (FID) of
3.31 and an Inception Score (IS) of 253.4. These results not only significantly
improve the state-of-the-art of class-unconditional image generation but also
rival the current leading methods in class-conditional image generation,
bridging the long-standing performance gap between these two tasks. Code is
available at https://github.com/LTH14/rcg.
RCG, a representation-conditioned image generation framework, achieves top performance in class-unconditional image generation using a self-supervised representation distribution and a representation diffusion model.
representation-conditioned class-unconditional image generation self-supervised representation pre-trained encoder representation diffusion model pixel generator high-quality image generation Frechet Inception Distance (FID) Inception Score (IS) class-conditional image generation