  Gaussian processes are flexible function approximators, with inductive biases
controlled by a covariance kernel. Learning the kernel is the key to
representation learning and strong predictive performance. In this paper, we
develop functional kernel learning (FKL) to directly infer functional
posteriors over kernels. In particular, we place a transformed Gaussian process
over a spectral density, to induce a non-parametric distribution over kernel
functions. The resulting approach enables learning of rich representations,
with support for any stationary kernel, uncertainty over the values of the
kernel, and an interpretable specification of a prior directly over kernels,
without requiring sophisticated initialization or manual intervention. We
perform inference through elliptical slice sampling, which is especially well
suited to marginalizing posteriors with the strongly correlated priors typical
to function space modelling. We develop our approach for non-uniform,
large-scale, multi-task, and multidimensional data, and show promising
performance in a wide range of settings, including interpolation,
extrapolation, and kernel recovery experiments.
