  Deep learning models trained on large-scale data have achieved encouraging
performance in many real-world tasks. Meanwhile, publishing those models
trained on sensitive datasets, such as medical records, could pose serious
privacy concerns. To counter these issues, one of the current state-of-the-art
approaches is the Private Aggregation of Teacher Ensembles, or PATE, which
achieved promising results in preserving the utility of the model while
providing a strong privacy guarantee. PATE combines an ensemble of "teacher
models" trained on sensitive data and transfers the knowledge to a "student"
model through the noisy aggregation of teachers' votes for labeling unlabeled
public data which the student model will be trained on. However, the knowledge
or voted labels learned by the student are noisy due to private aggregation.
Learning directly from noisy labels can significantly impact the accuracy of
the student model.
  In this paper, we propose the PATE++ mechanism, which combines the current
advanced noisy label training mechanisms with the original PATE framework to
enhance its accuracy. A novel structure of Generative Adversarial Nets (GANs)
is developed in order to integrate them effectively. In addition, we develop a
novel noisy label detection mechanism for semi-supervised model training to
further improve student model performance when training with noisy labels. We
evaluate our method on Fashion-MNIST and SVHN to show the improvements on the
original PATE on all measures.
