  Machine learning models leak information about their training data every time
they reveal a prediction. This is problematic when the training data needs to
remain private. Private prediction methods limit how much information about the
training data is leaked by each prediction. Private prediction can also be
achieved using models that are trained by private training methods. In private
prediction, both private training and private prediction methods exhibit
trade-offs between privacy, privacy failure probability, amount of training
data, and inference budget. Although these trade-offs are theoretically
well-understood, they have hardly been studied empirically. This paper presents
the first empirical study into the trade-offs of private prediction. Our study
sheds light on which methods are best suited for which learning setting.
Perhaps surprisingly, we find private training methods outperform private
prediction methods in a wide range of private prediction settings.
