  This paper discusses various theorems on the approximation capabilities of
neural networks (NNs), which are known as universal approximation theorems
(UATs). The paper gives a systematic overview of UATs starting from the
preliminary results on function approximation, such as Taylor's theorem,
Fourier's theorem, Weierstrass approximation theorem, Kolmogorov - Arnold
representation theorem, etc. Theoretical and numerical aspects of UATs are
covered from both arbitrary width and depth.
