  This paper studies minimax optimization problems defined over
infinite-dimensional function classes of overparameterized two-layer neural
networks. In particular, we consider the minimax optimization problem stemming
from estimating linear functional equations defined by conditional
expectations, where the objective functions are quadratic in the functional
spaces. We address (i) the convergence of the stochastic gradient
descent-ascent algorithm and (ii) the representation learning of the neural
networks. We establish convergence under the mean-field regime by considering
the continuous-time and infinite-width limit of the optimization dynamics.
Under this regime, the stochastic gradient descent-ascent corresponds to a
Wasserstein gradient flow over the space of probability measures defined over
the space of neural network parameters. We prove that the Wasserstein gradient
flow converges globally to a stationary point of the minimax objective at a
$O(T^{-1} + \alpha^{-1})$ sublinear rate, and additionally finds the solution
to the functional equation when the regularizer of the minimax objective is
strongly convex. Here $T$ denotes the time and $\alpha$ is a scaling parameter
of the neural networks. In terms of representation learning, our results show
that the feature representation induced by the neural networks is allowed to
deviate from the initial one by the magnitude of $O(\alpha^{-1})$, measured in
terms of the Wasserstein distance. Finally, we apply our general results to
concrete examples including policy evaluation, nonparametric instrumental
variable regression, asset pricing, and adversarial Riesz representer
estimation.
