  We show that the canonical approach for training differentially private GANs
-- updating the discriminator with differentially private stochastic gradient
descent (DPSGD) -- can yield significantly improved results after modifications
to training. Specifically, we propose that existing instantiations of this
approach neglect to consider how adding noise only to discriminator updates
inhibits discriminator training, disrupting the balance between the generator
and discriminator necessary for successful GAN training. We show that a simple
fix -- taking more discriminator steps between generator steps -- restores
parity between the generator and discriminator and improves results.
  Additionally, with the goal of restoring parity, we experiment with other
modifications -- namely, large batch sizes and adaptive discriminator update
frequency -- to improve discriminator training and see further improvements in
generation quality. Our results demonstrate that on standard image synthesis
benchmarks, DPSGD outperforms all alternative GAN privatization schemes. Code:
https://github.com/alexbie98/dpgan-revisit.
