  In generative adversarial networks, improving discriminators is one of the
key components for generation performance. As image classifiers are biased
toward texture and debiasing improves accuracy, we investigate 1) if the
discriminators are biased, and 2) if debiasing the discriminators will improve
generation performance. Indeed, we find empirical evidence that the
discriminators are sensitive to the style (e.g., texture and color) of images.
As a remedy, we propose feature statistics mixing regularization (FSMR) that
encourages the discriminator's prediction to be invariant to the styles of
input images. Specifically, we generate a mixed feature of an original and a
reference image in the discriminator's feature space and we apply
regularization so that the prediction for the mixed feature is consistent with
the prediction for the original image. We conduct extensive experiments to
demonstrate that our regularization leads to reduced sensitivity to style and
consistently improves the performance of various GAN architectures on nine
datasets. In addition, adding FSMR to recently-proposed augmentation-based GAN
methods further improves image quality. Our code is available at
https://github.com/naver-ai/FSMR.
