  Gaussian processes provide a powerful probabilistic kernel learning
framework, which allows learning high quality nonparametric regression models
via methods such as Gaussian process regression. Nevertheless, the learning
phase of Gaussian process regression requires massive computations which are
not realistic for large datasets. In this paper, we present a Gauss-Legendre
quadrature based approach for scaling up Gaussian process regression via a low
rank approximation of the kernel matrix. We utilize the structure of the low
rank approximation to achieve effective hyperparameter learning, training and
prediction. Our method is very much inspired by the well-known random Fourier
features approach, which also builds low-rank approximations via numerical
integration. However, our method is capable of generating high quality
approximation to the kernel using an amount of features which is
poly-logarithmic in the number of training points, while similar guarantees
will require an amount that is at the very least linear in the number of
training points when random Fourier features. Furthermore, the structure of the
low-rank approximation that our method builds is subtly different from the one
generated by random Fourier features, and this enables much more efficient
hyperparameter learning. The utility of our method for learning with
low-dimensional datasets is demonstrated using numerical experiments.
