  Generative Adversarial Networks (GAN) are trained to generate sample images
of interest distribution. To this end, generator network of GAN learns implicit
distribution of real data set from the classification with candidate generated
samples. Recently, various GANs have suggested novel ideas for stable
optimizing of its networks. However, in real implementation, sometimes they
still represent a only narrow part of true distribution or fail to converge. We
assume this ill posed problem comes from poor gradient from objective function
of discriminator, which easily trap the generator in a bad situation. To
address this problem, we propose a mode penalty GAN combined with pre-trained
auto encoder for explicit representation of generated and real data samples in
the encoded space. In this space, we make a generator manifold to follow a real
manifold by finding entire modes of target distribution. In addition, penalty
for uncovered modes of target distribution is given to the generator which
encourages it to find overall target distribution. We demonstrate that applying
the proposed method to GANs helps generator's optimization becoming more stable
and having faster convergence through experimental evaluations.
