  We propose a novel GAN training scheme that can handle any level of labeling
in a unified manner. Our scheme introduces a form of artificial labeling that
can incorporate manually defined labels, when available, and induce an
alignment between them. To define the artificial labels, we exploit the
assumption that neural network generators can be trained more easily to map
nearby latent vectors to data with semantic similarities, than across separate
categories. We use generated data samples and their corresponding artificial
conditioning labels to train a classifier. The classifier is then used to
self-label real data. To boost the accuracy of the self-labeling, we also use
the exponential moving average of the classifier. However, because the
classifier might still make mistakes, especially at the beginning of the
training, we also refine the labels through self-attention, by using the
labeling of real data samples only when the classifier outputs a high
classification probability score. We evaluate our approach on CIFAR-10, STL-10
and SVHN, and show that both self-labeling and self-attention consistently
improve the quality of generated data. More surprisingly, we find that the
proposed scheme can even outperform class-conditional GANs.
