  Generative adversarial networks (GANs) are challenging to train stably, and a
promising remedy of injecting instance noise into the discriminator input has
not been very effective in practice. In this paper, we propose Diffusion-GAN, a
novel GAN framework that leverages a forward diffusion chain to generate
Gaussian-mixture distributed instance noise. Diffusion-GAN consists of three
components, including an adaptive diffusion process, a diffusion
timestep-dependent discriminator, and a generator. Both the observed and
generated data are diffused by the same adaptive diffusion process. At each
diffusion timestep, there is a different noise-to-data ratio and the
timestep-dependent discriminator learns to distinguish the diffused real data
from the diffused generated data. The generator learns from the discriminator's
feedback by backpropagating through the forward diffusion chain, whose length
is adaptively adjusted to balance the noise and data levels. We theoretically
show that the discriminator's timestep-dependent strategy gives consistent and
helpful guidance to the generator, enabling it to match the true data
distribution. We demonstrate the advantages of Diffusion-GAN over strong GAN
baselines on various datasets, showing that it can produce more realistic
images with higher stability and data efficiency than state-of-the-art GANs.
