  An important use of private data is to build machine learning classifiers.
While there is a burgeoning literature on differentially private classification
algorithms, we find that they are not practical in real applications due to two
reasons. First, existing differentially private classifiers provide poor
accuracy on real world datasets. Second, there is no known differentially
private algorithm for empirically evaluating the private classifier on a
private test dataset.
  In this paper, we develop differentially private algorithms that mirror real
world empirical machine learning workflows. We consider the private classifier
training algorithm as a blackbox. We present private algorithms for selecting
features that are input to the classifier. Though adding a preprocessing step
takes away some of the privacy budget from the actual classification process
(thus potentially making it noisier and less accurate), we show that our novel
preprocessing techniques significantly increase classifier accuracy on three
real-world datasets. We also present the first private algorithms for
empirically constructing receiver operating characteristic (ROC) curves on a
private test set.
