  Complex-valued neural networks are not a new concept, however, the use of
real-valued models has often been favoured over complex-valued models due to
difficulties in training and performance. When comparing real-valued versus
complex-valued neural networks, existing literature often ignores the number of
parameters, resulting in comparisons of neural networks with vastly different
sizes. We find that when real and complex neural networks of similar capacity
are compared, complex models perform equal to or slightly worse than
real-valued models for a range of real-valued classification tasks. The use of
complex numbers allows neural networks to handle noise on the complex plane.
When classifying real-valued data with a complex-valued neural network, the
imaginary parts of the weights follow their real parts. This behaviour is
indicative for a task that does not require a complex-valued model. We further
investigated this in a synthetic classification task. We can transfer many
activation functions from the real to the complex domain using different
strategies. The weight initialisation of complex neural networks, however,
remains a significant problem.
