  Generative adversarial networks (GANs) nowadays are capable of producing
images of incredible realism. One concern raised is whether the
state-of-the-art GAN's learned distribution still suffers from mode collapse,
and what to do if so. Existing diversity tests of samples from GANs are usually
conducted qualitatively on a small scale, and/or depends on the access to
original training data as well as the trained model parameters. This paper
explores to diagnose GAN intra-mode collapse and calibrate that, in a novel
black-box setting: no access to training data, nor the trained model
parameters, is assumed. The new setting is practically demanded, yet rarely
explored and significantly more challenging. As a first stab, we devise a set
of statistical tools based on sampling, that can visualize, quantify, and
rectify intra-mode collapse. We demonstrate the effectiveness of our proposed
diagnosis and calibration techniques, via extensive simulations and
experiments, on unconditional GAN image generation (e.g., face and vehicle).
Our study reveals that the intra-mode collapse is still a prevailing problem in
state-of-the-art GANs and the mode collapse is diagnosable and calibratable in
black-box settings. Our codes are available at:
https://github.com/VITA-Group/BlackBoxGANCollapse.
