  Two recently introduced criteria for estimation of generative models are both
based on a reduction to binary classification. Noise-contrastive estimation
(NCE) is an estimation procedure in which a generative model is trained to be
able to distinguish data samples from noise samples. Generative adversarial
networks (GANs) are pairs of generator and discriminator networks, with the
generator network learning to generate samples by attempting to fool the
discriminator network into believing its samples are real data. Both estimation
procedures use the same function to drive learning, which naturally raises
questions about how they are related to each other, as well as whether this
function is related to maximum likelihood estimation (MLE). NCE corresponds to
training an internal data model belonging to the {\em discriminator} network
but using a fixed generator network. We show that a variant of NCE, with a
dynamic generator network, is equivalent to maximum likelihood estimation.
Since pairing a learned discriminator with an appropriate dynamically selected
generator recovers MLE, one might expect the reverse to hold for pairing a
learned generator with a certain discriminator. However, we show that
recovering MLE for a learned generator requires departing from the
distinguishability game. Specifically:
  (i) The expected gradient of the NCE discriminator can be made to match the
expected gradient of
  MLE, if one is allowed to use a non-stationary noise distribution for NCE,
  (ii) No choice of discriminator network can make the expected gradient for
the GAN generator match that of MLE, and
  (iii) The existing theory does not guarantee that GANs will converge in the
non-convex case.
  This suggests that the key next step in GAN research is to determine whether
GANs converge, and if not, to modify their training algorithm to force
convergence.
