  Gaussian processes (GPs) provide a nonparametric representation of functions.
However, classical GP inference suffers from high computational cost and it is
difficult to design nonstationary GP priors in practice. In this paper, we
propose a sparse Gaussian process model, EigenGP, based on the Karhunen-Loeve
(KL) expansion of a GP prior. We use the Nystrom approximation to obtain data
dependent eigenfunctions and select these eigenfunctions by evidence
maximization. This selection reduces the number of eigenfunctions in our model
and provides a nonstationary covariance function. To handle nonlinear
likelihoods, we develop an efficient expectation propagation (EP) inference
algorithm, and couple it with expectation maximization for eigenfunction
selection. Because the eigenfunctions of a Gaussian kernel are associated with
clusters of samples - including both the labeled and unlabeled - selecting
relevant eigenfunctions enables EigenGP to conduct semi-supervised learning.
Our experimental results demonstrate improved predictive performance of EigenGP
over alternative state-of-the-art sparse GP and semisupervised learning methods
for regression, classification, and semisupervised classification.
