  Gaussian processes are a flexible Bayesian nonparametric modelling approach
that has been widely applied but poses computational challenges. To address the
poor scaling of exact inference methods, approximation methods based on sparse
Gaussian processes (SGP) are attractive. An issue faced by SGP, especially in
latent variable models, is the inefficient learning of the inducing inputs,
which leads to poor model prediction. We propose a regularization approach by
balancing the reconstruction performance of data and the approximation
performance of the model itself. This regularization improves both inference
and prediction performance. We extend this regularization approach into latent
variable models with SGPs and show that performing variational inference (VI)
on those models is equivalent to performing VI on a related empirical Bayes
model.
