  Machine learning models have shone in a variety of domains and attracted
increasing attention from both the security and the privacy communities. One
important yet worrying question is: Will training models under the differential
privacy (DP) constraint have an unfavorable impact on their adversarial
robustness? While previous works have postulated that privacy comes at the cost
of worse robustness, we give the first theoretical analysis to show that DP
models can indeed be robust and accurate, even sometimes more robust than their
naturally-trained non-private counterparts. We observe three key factors that
influence the privacy-robustness-accuracy tradeoff: (1) hyper-parameters for DP
optimizers are critical; (2) pre-training on public data significantly
mitigates the accuracy and robustness drop; (3) choice of DP optimizers makes a
difference. With these factors set properly, we achieve 90\% natural accuracy,
72\% robust accuracy ($+9\%$ than the non-private model) under $l_2(0.5)$
attack, and 69\% robust accuracy ($+16\%$ than the non-private model) with
pre-trained SimCLRv2 model under $l_\infty(4/255)$ attack on CIFAR10 with
$\epsilon=2$. In fact, we show both theoretically and empirically that DP
models are Pareto optimal on the accuracy-robustness tradeoff. Empirically, the
robustness of DP models is consistently observed across various datasets and
models. We believe our encouraging results are a significant step towards
training models that are private as well as robust.
