  Wasserstein GAN(WGAN) is a model that minimizes the Wasserstein distance
between a data distribution and sample distribution. Recent studies have
proposed stabilizing the training process for the WGAN and implementing the
Lipschitz constraint. In this study, we prove the local stability of optimizing
the simple gradient penalty $\mu$-WGAN(SGP $\mu$-WGAN) under suitable
assumptions regarding the equilibrium and penalty measure $\mu$. The measure
valued differentiation concept is employed to deal with the derivative of the
penalty terms, which is helpful for handling abstract singular measures with
lower dimensional support. Based on this analysis, we claim that penalizing the
data manifold or sample manifold is the key to regularizing the original WGAN
with a gradient penalty. Experimental results obtained with unintuitive penalty
measures that satisfy our assumptions are also provided to support our
theoretical results.
