  Numerically solving high-dimensional partial differential equations (PDEs) is
a major challenge. Conventional methods, such as finite difference methods, are
unable to solve high-dimensional PDEs due to the curse-of-dimensionality. A
variety of deep learning methods have been recently developed to try and solve
high-dimensional PDEs by approximating the solution using a neural network. In
this paper, we prove global convergence for one of the commonly-used deep
learning algorithms for solving PDEs, the Deep Galerkin Method (DGM). DGM
trains a neural network approximator to solve the PDE using stochastic gradient
descent. We prove that, as the number of hidden units in the single-layer
network goes to infinity (i.e., in the ``wide network limit"), the trained
neural network converges to the solution of an infinite-dimensional linear
ordinary differential equation (ODE). The PDE residual of the limiting
approximator converges to zero as the training time $\rightarrow \infty$. Under
mild assumptions, this convergence also implies that the neural network
approximator converges to the solution of the PDE. A closely related class of
deep learning methods for PDEs is Physics Informed Neural Networks (PINNs).
Using the same mathematical techniques, we can prove a similar global
convergence result for the PINN neural network approximators. Both proofs
require analyzing a kernel function in the limit ODE governing the evolution of
the limit neural network approximator. A key technical challenge is that the
kernel function, which is a composition of the PDE operator and the neural
tangent kernel (NTK) operator, lacks a spectral gap, therefore requiring a
careful analysis of its properties.
