  Physical modeling is critical for many modern science and engineering
applications. From a data science or machine learning perspective, where more
domain-agnostic, data-driven models are pervasive, physical knowledge -- often
expressed as differential equations -- is valuable in that it is complementary
to data, and it can potentially help overcome issues such as data sparsity,
noise, and inaccuracy. In this work, we propose a simple, yet powerful and
general framework -- AutoIP, for Automatically Incorporating Physics -- that
can integrate all kinds of differential equations into Gaussian Processes (GPs)
to enhance prediction accuracy and uncertainty quantification. These equations
can be linear or nonlinear, spatial, temporal, or spatio-temporal, complete or
incomplete with unknown source terms, and so on. Based on kernel
differentiation, we construct a GP prior to sample the values of the target
function, equation-related derivatives, and latent source functions, which are
all jointly from a multivariate Gaussian distribution. The sampled values are
fed to two likelihoods: one to fit the observations, and the other to conform
to the equation. We use the whitening method to evade the strong dependency
between the sampled function values and kernel parameters, and we develop a
stochastic variational learning algorithm. AutoIP shows improvement upon
vanilla GPs in both simulation and several real-world applications, even using
rough, incomplete equations.
