  We introduce a new regression framework, Gaussian process regression networks
(GPRN), which combines the structural properties of Bayesian neural networks
with the non-parametric flexibility of Gaussian processes. This model
accommodates input dependent signal and noise correlations between multiple
response variables, input dependent length-scales and amplitudes, and
heavy-tailed predictive distributions. We derive both efficient Markov chain
Monte Carlo and variational Bayes inference procedures for this model. We apply
GPRN as a multiple output regression and multivariate volatility model,
demonstrating substantially improved performance over eight popular multiple
output (multi-task) Gaussian process models and three multivariate volatility
models on benchmark datasets, including a 1000 dimensional gene expression
dataset.
