  This paper presents an efficient variational inference framework for deriving
a family of structured gaussian process regression network (SGPRN) models. The
key idea is to incorporate auxiliary inducing variables in latent functions and
jointly treats both the distributions of the inducing variables and
hyper-parameters as variational parameters. Then we propose structured variable
distributions and marginalize latent variables, which enables the
decomposability of a tractable variational lower bound and leads to stochastic
optimization. Our inference approach is able to model data in which outputs do
not share a common input set with a computational complexity independent of the
size of the inputs and outputs and thus easily handle datasets with missing
values. We illustrate the performance of our method on synthetic data and real
datasets and show that our model generally provides better imputation results
on missing data than the state-of-the-art. We also provide a visualization
approach for time-varying correlation across outputs in electrocoticography
data and those estimates provide insight to understand the neural population
dynamics.
