  Generative adversarial networks (GAN) are a powerful subclass of generative
models. Despite a very rich research activity leading to numerous interesting
GAN algorithms, it is still very hard to assess which algorithm(s) perform
better than others. We conduct a neutral, multi-faceted large-scale empirical
study on state-of-the art models and evaluation measures. We find that most
models can reach similar scores with enough hyperparameter optimization and
random restarts. This suggests that improvements can arise from a higher
computational budget and tuning more than fundamental algorithmic changes. To
overcome some limitations of the current metrics, we also propose several data
sets on which precision and recall can be computed. Our experimental results
suggest that future GAN research should be based on more systematic and
objective evaluation procedures. Finally, we did not find evidence that any of
the tested algorithms consistently outperforms the non-saturating GAN
introduced in \cite{goodfellow2014generative}.
