  We examine two key questions in GAN training, namely overfitting and mode
drop, from an empirical perspective. We show that when stochasticity is removed
from the training procedure, GANs can overfit and exhibit almost no mode drop.
Our results shed light on important characteristics of the GAN training
procedure. They also provide evidence against prevailing intuitions that GANs
do not memorize the training set, and that mode dropping is mainly due to
properties of the GAN objective rather than how it is optimized during
training.
