  Due to the unstable nature of minimax game between generator and
discriminator, improving the performance of GANs is a challenging task. Recent
studies have shown that selected high-quality samples in training improve the
performance of GANs. However, sampling approaches which discard samples show
limitations in some aspects such as the speed of training and optimality of the
networks. In this paper we propose unrealistic feature suppression (UFS) module
that keeps high-quality features and suppresses unrealistic features. UFS
module keeps the training stability of networks and improves the quality of
generated images. We demonstrate the effectiveness of UFS module on various
models such as WGAN-GP, SNGAN, and BigGAN. By using UFS module, we achieved
better Frechet inception distance and inception score compared to various
baseline models. We also visualize how effectively our UFS module suppresses
unrealistic features through class activation maps.
