  A graph neural network transforms features in each vertex's neighborhood into
a vector representation of the vertex. Afterward, each vertex's representation
is used independently for predicting its label. This standard pipeline
implicitly assumes that vertex labels are conditionally independent given their
neighborhood features. However, this is a strong assumption, and we show that
it is far from true on many real-world graph datasets. Focusing on regression
tasks, we find that this conditional independence assumption severely limits
predictive power. This should not be that surprising, given that traditional
graph-based semi-supervised learning methods such as label propagation work in
the opposite fashion by explicitly modeling the correlation in predicted
outcomes.
  Here, we address this problem with an interpretable and efficient framework
that can improve any graph neural network architecture simply by exploiting
correlation structure in the regression residuals. In particular, we model the
joint distribution of residuals on vertices with a parameterized multivariate
Gaussian, and estimate the parameters by maximizing the marginal likelihood of
the observed labels. Our framework achieves substantially higher accuracy than
competing baselines, and the learned parameters can be interpreted as the
strength of correlation among connected vertices. Furthermore, we develop
linear time algorithms for low-variance, unbiased model parameter estimates,
allowing us to scale to large networks. We also provide a basic version of our
method that makes stronger assumptions on correlation structure but is painless
to implement, often leading to great practical performance with minimal
overhead.
