  Generative adversarial networks (GAN) approximate a target data distribution
by jointly optimizing an objective function through a "two-player game" between
a generator and a discriminator. Despite their empirical success, however, two
very basic questions on how well they can approximate the target distribution
remain unanswered. First, it is not known how restricting the discriminator
family affects the approximation quality. Second, while a number of different
objective functions have been proposed, we do not understand when convergence
to the global minima of the objective function leads to convergence to the
target distribution under various notions of distributional convergence.
  In this paper, we address these questions in a broad and unified setting by
defining a notion of adversarial divergences that includes a number of recently
proposed objective functions. We show that if the objective function is an
adversarial divergence with some additional conditions, then using a restricted
discriminator family has a moment-matching effect. Additionally, we show that
for objective functions that are strict adversarial divergences, convergence in
the objective function implies weak convergence, thus generalizing previous
results.
