  It has been shown that gradient descent can yield the zero training loss in
the over-parametrized regime (the width of the neural networks is much larger
than the number of data points). In this work, combining the ideas of some
existing works, we investigate the gradient descent method for training
two-layer neural networks for approximating some target continuous functions.
By making use the generic chaining technique from probability theory, we show
that gradient descent can yield an exponential convergence rate, while the
width of the neural networks needed is independent of the size of the training
data. The result also implies some strong approximation ability of the
two-layer neural networks without curse of dimensionality.
