  Choosing a proper set of kernel functions is an important problem in learning
Gaussian Process (GP) models since each kernel structure has different model
complexity and data fitness. Recently, automatic kernel composition methods
provide not only accurate prediction but also attractive interpretability
through search-based methods. However, existing methods suffer from slow kernel
composition learning. To tackle large-scaled data, we propose a new sparse
approximate posterior for GPs, MultiSVGP, constructed from groups of inducing
points associated with individual additive kernels in compositional kernels. We
demonstrate that this approximation provides a better fit to learn
compositional kernels given empirical observations. We also provide
theoretically justification on error bound when compared to the traditional
sparse GP. In contrast to the search-based approach, we present a novel
probabilistic algorithm to learn a kernel composition by handling the sparsity
in the kernel selection with Horseshoe prior. We demonstrate that our model can
capture characteristics of time series with significant reductions in
computational time and have competitive regression performance on real-world
data sets.
