  Despite recent theoretical progress on the non-convex optimization of
two-layer neural networks, it is still an open question whether gradient
descent on neural networks without unnatural modifications can achieve better
sample complexity than kernel methods. This paper provides a clean mean-field
analysis of projected gradient flow on polynomial-width two-layer neural
networks. Different from prior works, our analysis does not require unnatural
modifications of the optimization algorithm. We prove that with sample size $n
= O(d^{3.1})$ where $d$ is the dimension of the inputs, the network trained
with projected gradient flow converges in $\text{poly}(d)$ time to a
non-trivial error that is not achievable by kernel methods using $n \ll d^4$
samples, hence demonstrating a clear separation between unmodified gradient
descent and NTK. As a corollary, we show that projected gradient descent with a
positive learning rate and a polynomial number of iterations converges to low
error with the same sample complexity.
