  In this paper, we propose a novel normalization method called penalty
gradient normalization (PGN) to tackle the training instability of Generative
Adversarial Networks (GANs) caused by the sharp gradient space. Unlike existing
work such as gradient penalty and spectral normalization, the proposed PGN only
imposes a penalty gradient norm constraint on the discriminator function, which
increases the capacity of the discriminator. Moreover, the proposed penalty
gradient normalization can be applied to different GAN architectures with
little modification. Extensive experiments on three datasets show that GANs
trained with penalty gradient normalization outperform existing methods in
terms of both Frechet Inception and Distance and Inception Score.
