  We introduce a simple, rigorous, and unified framework for solving nonlinear
partial differential equations (PDEs), and for solving inverse problems (IPs)
involving the identification of parameters in PDEs, using the framework of
Gaussian processes. The proposed approach: (1) provides a natural
generalization of collocation kernel methods to nonlinear PDEs and IPs; (2) has
guaranteed convergence for a very general class of PDEs, and comes equipped
with a path to compute error bounds for specific PDE approximations; (3)
inherits the state-of-the-art computational complexity of linear solvers for
dense kernel matrices. The main idea of our method is to approximate the
solution of a given PDE as the maximum a posteriori (MAP) estimator of a
Gaussian process conditioned on solving the PDE at a finite number of
collocation points. Although this optimization problem is infinite-dimensional,
it can be reduced to a finite-dimensional one by introducing additional
variables corresponding to the values of the derivatives of the solution at
collocation points; this generalizes the representer theorem arising in
Gaussian process regression. The reduced optimization problem has the form of a
quadratic objective function subject to nonlinear constraints; it is solved
with a variant of the Gauss--Newton method. The resulting algorithm (a) can be
interpreted as solving successive linearizations of the nonlinear PDE, and (b)
in practice is found to converge in a small number of iterations (2 to 10), for
a wide range of PDEs. Most traditional approaches to IPs interleave parameter
updates with numerical solution of the PDE; our algorithm solves for both
parameter and PDE solution simultaneously. Experiments on nonlinear elliptic
PDEs, Burgers' equation, a regularized Eikonal equation, and an IP for
permeability identification in Darcy flow illustrate the efficacy and scope of
our framework.
