  Lipschitz continuity recently becomes popular in generative adversarial
networks (GANs). It was observed that the Lipschitz regularized discriminator
leads to improved training stability and sample quality. The mainstream
implementations of Lipschitz continuity include gradient penalty and spectral
normalization. In this paper, we demonstrate that gradient penalty introduces
undesired bias, while spectral normalization might be over restrictive. We
accordingly propose a new method which is efficient and unbiased. Our
experiments verify our analysis and show that the proposed method is able to
achieve successful training in various situations where gradient penalty and
spectral normalization fail.
