  One obstacle to the use of Gaussian processes (GPs) in large-scale problems,
and as a component in deep learning system, is the need for bespoke derivations
and implementations for small variations in the model or inference. In order to
improve the utility of GPs we need a modular system that allows rapid
implementation and testing, as seen in the neural network community. We present
a mathematical and software framework for scalable approximate inference in
GPs, which combines interdomain approximations and multiple outputs. Our
framework, implemented in GPflow, provides a unified interface for many
existing multioutput models, as well as more recent convolutional structures.
This simplifies the creation of deep models with GPs, and we hope that this
work will encourage more interest in this approach.
