  Over the past few years, there has been growing interest in developing larger
and deeper neural networks, including deep generative models like generative
adversarial networks (GANs). However, GANs typically come with high
computational complexity, leading researchers to explore methods for reducing
the training and inference costs. One such approach gaining popularity in
supervised learning is dynamic sparse training (DST), which maintains good
performance while enjoying excellent training efficiency. Despite its potential
benefits, applying DST to GANs presents challenges due to the adversarial
nature of the training process. In this paper, we propose a novel metric called
the balance ratio (BR) to study the balance between the sparse generator and
discriminator. We also introduce a new method called balanced dynamic sparse
training (ADAPT), which seeks to control the BR during GAN training to achieve
a good trade-off between performance and computational cost. Our proposed
method shows promising results on multiple datasets, demonstrating its
effectiveness.
