  We consider optimizing two-layer neural networks in the mean-field regime
where the learning dynamics of network weights can be approximated by the
evolution in the space of probability measures over the weight parameters
associated with the neurons. The mean-field regime is a theoretically
attractive alternative to the NTK (lazy training) regime which is only
restricted locally in the so-called neural tangent kernel space around
specialized initializations. Several prior works (\cite{chizat2018global,
mei2018mean}) establish the asymptotic global optimality of the mean-field
regime, but it is still challenging to obtain a quantitative convergence rate
due to the complicated unbounded nonlinearity of the training dynamics. This
work establishes the first linear convergence result for vanilla two-layer
neural networks trained by continuous-time noisy gradient descent in the
mean-field regime. Our result relies on a novel time-depdendent estimate of the
logarithmic Sobolev constants for a family of measures determined by the
evolving distribution of hidden neurons.
