  Inference for GP models with non-Gaussian noises is computationally expensive
when dealing with large datasets. Many recent inference methods approximate the
posterior distribution with a simpler distribution defined on a small number of
inducing points. The inference is accurate only when data points have strong
correlation with these inducing points. In this paper, we consider the
inference problem in a different direction: GP function values in the posterior
are mostly correlated in short distance. We construct a variational
distribution such that the inference for a data point considers only its
neighborhood. With this construction, the variational lower bound is highly
decomposible, hence we can run stochastic optimization with very small batches.
We then train Graph Convolutional Networks as a reusable model to identify
variational parameters for each data point. Model reuse greatly reduces the
number of parameters and the number of iterations needed in optimization. The
proposed method significantly speeds up the inference and often gets more
accurate results than previous methods.
