  Gaussian Processes (GPs) are a generic modelling tool for supervised
learning. While they have been successfully applied on large datasets, their
use in safety-critical applications is hindered by the lack of good performance
guarantees. To this end, we propose a method to learn GPs and their sparse
approximations by directly optimizing a PAC-Bayesian bound on their
generalization performance, instead of maximizing the marginal likelihood.
Besides its theoretical appeal, we find in our evaluation that our learning
method is robust and yields significantly better generalization guarantees than
other common GP approaches on several regression benchmark datasets.
