  In this paper, we investigate the underlying factor that leads to failure and
success in the training of GANs. We study the property of the optimal
discriminative function and show that in many GANs, the gradient from the
optimal discriminative function is not reliable, which turns out to be the
fundamental cause of failure in training of GANs. We further demonstrate that a
well-defined distance metric does not necessarily guarantee the convergence of
GANs. Finally, we prove in this paper that Lipschitz-continuity condition is a
general solution to make the gradient of the optimal discriminative function
reliable, and characterized the necessary condition where Lipschitz-continuity
ensures the convergence, which leads to a broad family of valid GAN objectives
under Lipschitz-continuity condition, where Wasserstein distance is one special
case. We experiment with several new objectives, which are sound according to
our theorems, and we found that, compared with Wasserstein distance, the
outputs of the discriminator with new objectives are more stable and the final
qualities of generated samples are also consistently higher than those produced
by Wasserstein distance.
