  This paper introduces a novel and fully unsupervised framework for
conditional GAN training in which labels are automatically obtained from data.
We incorporate a clustering network into the standard conditional GAN framework
that plays against the discriminator. With the generator, it aims to find a
shared structured mapping for associating pseudo-labels with the real and fake
images. Our generator outperforms unconditional GANs in terms of FID with
significant margins on large scale datasets like ImageNet and LSUN. It also
outperforms class conditional GANs trained on human labels on CIFAR10 and
CIFAR100 where fine-grained annotations or a large number of samples per class
are not available. Additionally, our clustering network exceeds the
state-of-the-art on CIFAR100 clustering.
