  Marginalising over families of Gaussian Process kernels produces flexible
model classes with well-calibrated uncertainty estimates. Existing approaches
require likelihood evaluations of many kernels, rendering them prohibitively
expensive for larger datasets. We propose a Bayesian Quadrature scheme to make
this marginalisation more efficient and thereby more practical. Through use of
the maximum mean discrepancies between distributions, we define a kernel over
kernels that captures invariances between Spectral Mixture (SM) Kernels. Kernel
samples are selected by generalising an information-theoretic acquisition
function for warped Bayesian Quadrature. We show that our framework achieves
more accurate predictions with better calibrated uncertainty than
state-of-the-art baselines, especially when given limited (wall-clock) time
budgets.
