  The vanilla GAN (Goodfellow et al. 2014) suffers from mode collapse deeply,
which usually manifests as that the images generated by generators tend to have
a high similarity amongst them, even though their corresponding latent vectors
have been very different. In this paper, we introduce a pluggable diversity
penalty module (DPM) to alleviate mode collapse of GANs. It reduces the
similarity of image pairs in feature space, i.e., if two latent vectors are
different, then we enforce the generator to generate two images with different
features. The normalized Gram matrix is used to measure the similarity. We
compare the proposed method with Unrolled GAN (Metz et al. 2016), BourGAN
(Xiao, Zhong, and Zheng 2018), PacGAN (Lin et al. 2018), VEEGAN (Srivastava et
al. 2017) and ALI (Dumoulin et al. 2016) on 2D synthetic dataset, and results
show that the diversity penalty module can help GAN capture much more modes of
the data distribution. Further, in classification tasks, we apply this method
as image data augmentation on MNIST, Fashion- MNIST and CIFAR-10, and the
classification testing accuracy is improved by 0.24%, 1.34% and 0.52% compared
with WGAN GP (Gulrajani et al. 2017), respectively. In domain translation,
diversity penalty module can help StarGAN (Choi et al. 2018) generate more
accurate attention masks and accelarate the convergence process. Finally, we
quantitatively evaluate the proposed method with IS and FID on CelebA,
CIFAR-10, MNIST and Fashion-MNIST, and the results suggest GAN with diversity
penalty module gets much higher IS and lower FID compared with some SOTA GAN
architectures.
