  Generative Adversarial Networks (GANs) are among the most popular approaches
to generate synthetic data, especially images, for data sharing purposes. Given
the vital importance of preserving the privacy of the individual data points in
the original data, GANs are trained utilizing frameworks with robust privacy
guarantees such as Differential Privacy (DP). However, these approaches remain
widely unstudied beyond single performance metrics when presented with
imbalanced datasets. To this end, we systematically compare GANs trained with
the two best-known DP frameworks for deep learning, DP-SGD, and PATE, in
different data imbalance settings from two perspectives -- the size of the
classes in the generated synthetic data and their classification performance.
  Our analyses show that applying PATE, similarly to DP-SGD, has a disparate
effect on the under/over-represented classes but in a much milder magnitude
making it more robust. Interestingly, our experiments consistently show that
for PATE, unlike DP-SGD, the privacy-utility trade-off is not monotonically
decreasing but is much smoother and inverted U-shaped, meaning that adding a
small degree of privacy actually helps generalization. However, we have also
identified some settings (e.g., large imbalance) where PATE-GAN completely
fails to learn some subparts of the training data.
