  Linear systems occur throughout engineering and the sciences, most notably as
differential equations. In many cases the forcing function for the system is
unknown, and interest lies in using noisy observations of the system to infer
the forcing, as well as other unknown parameters. In differential equations,
the forcing function is an unknown function of the independent variables
(typically time and space), and can be modelled as a Gaussian process (GP). In
this paper we show how the adjoint of a linear system can be used to
efficiently infer forcing functions modelled as GPs, using a truncated basis
expansion of the GP kernel. We show how exact conjugate Bayesian inference for
the truncated GP can be achieved, in many cases with substantially lower
computation than would be required using MCMC methods. We demonstrate the
approach on systems of both ordinary and partial differential equations, and
show that the basis expansion approach approximates well the true forcing with
a modest number of basis vectors. Finally, we show how to infer point estimates
for the non-linear model parameters, such as the kernel length-scales, using
Bayesian optimisation.
