  We show that differentially private stochastic gradient descent (DP-SGD) can
yield poorly calibrated, overconfident deep learning models. This represents a
serious issue for safety-critical applications, e.g. in medical diagnosis. We
highlight and exploit parallels between stochastic gradient Langevin dynamics,
a scalable Bayesian inference technique for training deep neural networks, and
DP-SGD, in order to train differentially private, Bayesian neural networks with
minor adjustments to the original (DP-SGD) algorithm. Our approach provides
considerably more reliable uncertainty estimates than DP-SGD, as demonstrated
empirically by a reduction in expected calibration error (MNIST $\sim{5}$-fold,
Pediatric Pneumonia Dataset $\sim{2}$-fold).
