  In this paper, we propose a novel loss function for training Generative
Adversarial Networks (GANs) aiming towards deeper theoretical understanding as
well as improved stability and performance for the underlying optimization
problem. The new loss function is based on cumulant generating functions giving
rise to \emph{Cumulant GAN}. Relying on a recently-derived variational formula,
we show that the corresponding optimization problem is equivalent to R{\'e}nyi
divergence minimization, thus offering a (partially) unified perspective of GAN
losses: the R{\'e}nyi family encompasses Kullback-Leibler divergence (KLD),
reverse KLD, Hellinger distance and $\chi^2$-divergence. Wasserstein GAN is
also a member of cumulant GAN. In terms of stability, we rigorously prove the
linear convergence of cumulant GAN to the Nash equilibrium for a linear
discriminator, Gaussian distributions and the standard gradient descent ascent
algorithm. Finally, we experimentally demonstrate that image generation is more
robust relative to Wasserstein GAN and it is substantially improved in terms of
both inception score and Fr\'echet inception distance when both weaker and
stronger discriminators are considered.
