  The empirical success of Generative Adversarial Networks (GANs) caused an
increasing interest in theoretical research. The statistical literature is
mainly focused on Wasserstein GANs and generalizations thereof, which
especially allow for good dimension reduction properties. Statistical results
for Vanilla GANs, the original optimization problem, are still rather limited
and require assumptions such as smooth activation functions and equal
dimensions of the latent space and the ambient space. To bridge this gap, we
draw a connection from Vanilla GANs to the Wasserstein distance. By doing so,
existing results for Wasserstein GANs can be extended to Vanilla GANs. In
particular, we obtain an oracle inequality for Vanilla GANs in Wasserstein
distance. The assumptions of this oracle inequality are designed to be
satisfied by network architectures commonly used in practice, such as
feedforward ReLU networks. By providing a quantitative result for the
approximation of a Lipschitz function by a feedforward ReLU network with
bounded H\"older norm, we conclude a rate of convergence for Vanilla GANs as
well as Wasserstein GANs as estimators of the unknown probability distribution.
