  Generative Adversarial Networks (GANs) have been widely applied in different
scenarios thanks to the development of deep neural networks. The original GAN
was proposed based on the non-parametric assumption of the infinite capacity of
networks. However, it is still unknown whether GANs can fit the target
distribution without any prior information. Due to the overconfident
assumption, many issues remain unaddressed in GANs' training, such as
non-convergence, mode collapses, gradient vanishing. Regularization and
normalization are common methods of introducing prior information to stabilize
training and improve discrimination. Although a handful number of
regularization and normalization methods have been proposed for GANs, to the
best of our knowledge, there exists no comprehensive survey that primarily
focuses on objectives and development of these methods, apart from some
in-comprehensive and limited scope studies. In this work, we conduct a
comprehensive survey on the regularization and normalization techniques from
different perspectives of GANs training. First, we systematically describe
different perspectives of GANs training and thus obtain the different
objectives of regularization and normalization. Based on these objectives, we
propose a new taxonomy. Furthermore, we compare the performance of the
mainstream methods on different datasets and investigate the applications of
regularization and normalization techniques that have been frequently employed
in state-of-the-art GANs. Finally, we highlight potential future directions of
research in this domain. Code and studies related to the regularization and
normalization of GANs in this work is summarized on
https://github.com/iceli1007/GANs-Regularization-Review.
