  Inspired by recent advances in the field of expert-based approximations of
Gaussian processes (GPs), we present an expert-based approach to large-scale
multi-output regression using single-output GP experts. Employing a deeply
structured mixture of single-output GPs encoded via a probabilistic circuit
allows us to capture correlations between multiple output dimensions
accurately. By recursively partitioning the covariate space and the output
space, posterior inference in our model reduces to inference on single-output
GP experts, which only need to be conditioned on a small subset of the
observations. We show that inference can be performed exactly and efficiently
in our model, that it can capture correlations between output dimensions and,
hence, often outperforms approaches that do not incorporate inter-output
correlations, as demonstrated on several data sets in terms of the negative log
predictive density.
