PhD Student, City University of Hong Kong
An artificial intelligence algorithms used in unsupervised learning
GANs is used to learn an implicit distribution (e.g., images) from the data. The basic idea of GANs is to simultaneously train a discriminator and a generator: the discriminator aims to distinguish between real samples and generated samples; while the generator tries to generate fake samples as real as possible, making the discriminator believe that the fake samples are from real data. GANs have demonstrated impressive performance for various computer vision tasks such as image generation, image super-resolution, and semi-supervised learning.
In spite of the great progress for GANs in image generation, GANs still face two problems. The first one is that the quality of generated images is still limited for some realistic tasks. The second one is that the learning process of GANs may be unstable. We found that these problems are partially caused by the loss function of the original GANs. The original GANs hypothesize the discriminator as a classifier with the sigmoid cross entropy loss function, which may lead to the vanishing gradients problem during the learning process. To overcome such a problem, we propose the Least Squares Generative Adversarial Networks (LSGANs) which adopt the least squares loss function for the discriminator. We show that minimizing the objective function of LSGAN yields minimizing the Pearson Chi^2 divergence. There are two benefits of LSGANs over original GANs. First, LSGANs are able to generate higher quality images than original GANs. Second, LSGANs perform more stable during the learning process. We evaluate LSGANs on several datasets and the experimental results show that the images generated by LSGANs are of better quality than the ones generated by original GANs. We also conduct two comparison experiments between LSGANs and original GANs to illustrate the stability of LSGANs.
Abstract: This report summarizes the tutorial presented by the author at NIPS 2016 on generative adversarial networks (GANs). The tutorial describes: (1) Why generative modeling is a topic worth studying, (2) how generative models work, and how GANs compare to other generative models, (3) the details of how GANs work, (4) research frontiers in GANs, and (5) state-of-the-art image models that combine GANs with other methods. Finally, the tutorial contains three exercises for readers to complete, and the solutions to these exercises.
Pub.: 31 Dec '16, Pinned: 24 Aug '17
Abstract: We present a variety of new architectural features and training procedures that we apply to the generative adversarial networks (GANs) framework. We focus on two applications of GANs: semi-supervised learning, and the generation of images that humans find visually realistic. Unlike most work on generative models, our primary goal is not to train a model that assigns high likelihood to test data, nor do we require the model to be able to learn well without using any labels. Using our new techniques, we achieve state-of-the-art results in semi-supervised classification on MNIST, CIFAR-10 and SVHN. The generated images are of high quality as confirmed by a visual Turing test: our model generates MNIST samples that humans cannot distinguish from real data, and CIFAR-10 samples that yield a human error rate of 21.3%. We also present ImageNet samples with unprecedented resolution and show that our methods enable the model to learn recognizable features of ImageNet classes.
Pub.: 10 Jun '16, Pinned: 24 Aug '17
Abstract: Generative adversarial networks (GANs) have achieved huge success in unsupervised learning. Most of GANs treat the discriminator as a classifier with the binary sigmoid cross entropy loss function. However, we find that the sigmoid cross entropy loss function will sometimes lead to the saturation problem in GANs learning. In this work, we propose to adopt the L2 loss function for the discriminator. The properties of the L2 loss function can improve the stabilization of GANs learning. With the usage of the L2 loss function, we propose the multi-class generative adversarial networks for the purpose of image generation with multiple classes. We evaluate the multi-class GANs on a handwritten Chinese characters dataset with 3740 classes. The experiments demonstrate that the multi-class GANs can generate elegant images on datasets with a large number of classes. Comparison experiments between the L2 loss function and the sigmoid cross entropy loss function are also conducted and the results demonstrate the stabilization of the L2 loss function.
Pub.: 12 Nov '16, Pinned: 24 Aug '17
Abstract: In recent years, supervised learning with convolutional networks (CNNs) has seen huge adoption in computer vision applications. Comparatively, unsupervised learning with CNNs has received less attention. In this work we hope to help bridge the gap between the success of CNNs for supervised learning and unsupervised learning. We introduce a class of CNNs called deep convolutional generative adversarial networks (DCGANs), that have certain architectural constraints, and demonstrate that they are a strong candidate for unsupervised learning. Training on various image datasets, we show convincing evidence that our deep convolutional adversarial pair learns a hierarchy of representations from object parts to scenes in both the generator and discriminator. Additionally, we use the learned features for novel tasks - demonstrating their applicability as general image representations.
Pub.: 07 Jan '16, Pinned: 24 Aug '17
Join Sparrho today to stay on top of science
Discover, organise and share research that matters to you