  Momentum methods, such as heavy ball method~(HB) and Nesterov's accelerated
gradient method~(NAG), have been widely used in training neural networks by
incorporating the history of gradients into the current updating process. In
practice, they often provide improved performance over (stochastic) gradient
descent~(GD) with faster convergence. Despite these empirical successes,
theoretical understandings of their accelerated convergence rates are still
lacking. Recently, some attempts have been made by analyzing the trajectories
of gradient-based methods in an over-parameterized regime, where the number of
the parameters is significantly larger than the number of the training
instances. However, the majority of existing theoretical work is mainly
concerned with GD and the established convergence result of NAG is inferior to
HB and GD, which fails to explain the practical success of NAG. In this paper,
we take a step towards closing this gap by analyzing NAG in training a randomly
initialized over-parameterized two-layer fully connected neural network with
ReLU activation. Despite the fact that the objective function is non-convex and
non-smooth, we show that NAG converges to a global minimum at a non-asymptotic
linear rate $(1-\Theta(1/\sqrt{\kappa}))^t$, where $\kappa > 1$ is the
condition number of a gram matrix and $t$ is the number of the iterations.
Compared to the convergence rate $(1-\Theta(1/{\kappa}))^t$ of GD, our result
provides theoretical guarantees for the acceleration of NAG in neural network
training. Furthermore, our findings suggest that NAG and HB have similar
convergence rate. Finally, we conduct extensive experiments on six benchmark
datasets to validate the correctness of our theoretical results.
