  The instability in GAN training has been a long-standing problem despite
remarkable research efforts. We identify that instability issues stem from
difficulties of performing feature matching with mini-batch statistics, due to
a fragile balance between the fixed target distribution and the progressively
generated distribution. In this work, we propose Feature Quantization (FQ) for
the discriminator, to embed both true and fake data samples into a shared
discrete space. The quantized values of FQ are constructed as an evolving
dictionary, which is consistent with feature statistics of the recent
distribution history. Hence, FQ implicitly enables robust feature matching in a
compact space. Our method can be easily plugged into existing GAN models, with
little computational overhead in training. We apply FQ to 3 representative GAN
models on 9 benchmarks: BigGAN for image generation, StyleGAN for face
synthesis, and U-GAT-IT for unsupervised image-to-image translation. Extensive
experimental results show that the proposed FQ-GAN can improve the FID scores
of baseline methods by a large margin on a variety of tasks, achieving new
state-of-the-art performance.
