  Differentially Private Stochastic Gradient Descent (DP-SGD) is a key method
for applying privacy in the training of deep learning models. It applies
isotropic Gaussian noise to gradients during training, which can perturb these
gradients in any direction, damaging utility. Metric DP, however, can provide
alternative mechanisms based on arbitrary metrics that might be more suitable
for preserving utility. In this paper, we apply \textit{directional privacy},
via a mechanism based on the von Mises-Fisher (VMF) distribution, to perturb
gradients in terms of \textit{angular distance} so that gradient direction is
broadly preserved. We show that this provides both $\epsilon$-DP and $\epsilon
d$-privacy for deep learning training, rather than the $(\epsilon,
\delta)$-privacy of the Gaussian mechanism. Experiments on key datasets then
indicate that the VMF mechanism can outperform the Gaussian in the
utility-privacy trade-off. In particular, our experiments provide a direct
empirical comparison of privacy between the two approaches in terms of their
ability to defend against reconstruction and membership inference.
