  We present $\zeta$-DP, an extension of differential privacy (DP) to
complex-valued functions. After introducing the complex Gaussian mechanism,
whose properties we characterise in terms of $(\varepsilon, \delta)$-DP and
R\'enyi-DP, we present $\zeta$-DP stochastic gradient descent ($\zeta$-DP-SGD),
a variant of DP-SGD for training complex-valued neural networks. We
experimentally evaluate $\zeta$-DP-SGD on three complex-valued tasks, i.e.
electrocardiogram classification, speech classification and magnetic resonance
imaging (MRI) reconstruction. Moreover, we provide $\zeta$-DP-SGD benchmarks
for a large variety of complex-valued activation functions and on a
complex-valued variant of the MNIST dataset. Our experiments demonstrate that
DP training of complex-valued neural networks is possible with rigorous privacy
guarantees and excellent utility.
