  Generative adversarial networks (GANs) have enjoyed much success in learning
high-dimensional distributions. Learning objectives approximately minimize an
$f$-divergence ($f$-GANs) or an integral probability metric (Wasserstein GANs)
between the model and the data distribution using a discriminator. Wasserstein
GANs enjoy superior empirical performance, but in $f$-GANs the discriminator
can be interpreted as a density ratio estimator which is necessary in some GAN
applications. In this paper, we bridge the gap between $f$-GANs and Wasserstein
GANs (WGANs). First, we list two constraints over variational $f$-divergence
estimation objectives that preserves the optimal solution. Next, we minimize
over a Lagrangian relaxation of the constrained objective, and show that it
generalizes critic objectives of both $f$-GAN and WGAN. Based on this
generalization, we propose a novel practical objective, named KL-Wasserstein
GAN (KL-WGAN). We demonstrate empirical success of KL-WGAN on synthetic
datasets and real-world image generation benchmarks, and achieve
state-of-the-art FID scores on CIFAR10 image generation.
