  We introduce stochastic variational inference for Gaussian process models.
This enables the application of Gaussian process (GP) models to data sets
containing millions of data points. We show how GPs can be vari- ationally
decomposed to depend on a set of globally relevant inducing variables which
factorize the model in the necessary manner to perform variational inference.
Our ap- proach is readily extended to models with non-Gaussian likelihoods and
latent variable models based around Gaussian processes. We demonstrate the
approach on a simple toy problem and two real world data sets.
