  Generative adversarial networks (GANs) are one of the most powerful
generative models, but always require a large and balanced dataset to train.
Traditional GANs are not applicable to generate minority-class images in a
highly imbalanced dataset. Balancing GAN (BAGAN) is proposed to mitigate this
problem, but it is unstable when images in different classes look similar, e.g.
flowers and cells. In this work, we propose a supervised autoencoder with an
intermediate embedding model to disperse the labeled latent vectors. With the
improved autoencoder initialization, we also build an architecture of BAGAN
with gradient penalty (BAGAN-GP). Our proposed model overcomes the unstable
issue in original BAGAN and converges faster to high quality generations. Our
model achieves high performance on the imbalanced scale-down version of MNIST
Fashion, CIFAR-10, and one small-scale medical image dataset.
