  Complex-valued neural networks (CVNNs) have recently been successful in
various pioneering areas which involve wave-typed information and
frequency-domain processing. This work addresses different structures and
classification of CVNNs. The theory behind complex activation functions,
implications related to complex differentiability and special activations for
CVNN output layers are presented. The work also discusses CVNN learning and
optimization using gradient and non-gradient based algorithms. Complex
Backpropagation utilizing complex chain rule is also explained in terms of
Wirtinger calculus. Moreover, special modules for building CVNN models, such as
complex batch normalization and complex random initialization are also
discussed. The work also highlights libraries and software blocks proposed for
CVNN implementations and discusses future directions. The objective of this
work is to understand the dynamics and most recent developments of CVNNs.
