  Gaussian process models are flexible, Bayesian non-parametric approaches to
regression. Properties of multivariate Gaussians mean that they can be combined
linearly in the manner of additive models and via a link function (like in
generalized linear models) to handle non-Gaussian data. However, the link
function formalism is restrictive, link functions are always invertible and
must convert a parameter of interest to a linear combination of the underlying
processes. There are many likelihoods and models where a non-linear combination
is more appropriate. We term these more general models Chained Gaussian
Processes: the transformation of the GPs to the likelihood parameters will not
generally be invertible, and that implies that linearisation would only be
possible with multiple (localized) links, i.e. a chain. We develop an
approximate inference procedure for Chained GPs that is scalable and applicable
to any factorized likelihood. We demonstrate the approximation on a range of
likelihood functions.
