  Deep Neural Networks are widely used in academy as well as corporate and
public applications, including safety critical applications such as health care
and autonomous driving. The ability to explain their output is critical for
safety reasons as well as acceptance among applicants. A multitude of methods
have been proposed to explain real-valued neural networks. Recently,
complex-valued neural networks have emerged as a new class of neural networks
dealing with complex-valued input data without the necessity of projecting them
onto $\mathbb{R}^2$. This brings up the need to develop explanation algorithms
for this kind of neural networks. In this paper we provide these developments.
While we focus on adapting the widely used DeepSHAP algorithm to the complex
domain, we also present versions of four gradient based explanation methods
suitable for use in complex-valued neural networks. We evaluate the explanation
quality of all presented algorithms and provide all of them as an open source
library adaptable to most recent complex-valued neural network architectures.
