  Capturing global contextual representations by exploiting long-range
pixel-pixel dependencies has shown to improve semantic segmentation
performance. However, how to do this efficiently is an open question as current
approaches of utilising attention schemes or very deep models to increase the
models field of view, result in complex models with large memory consumption.
Inspired by recent work on graph neural networks, we propose the
Self-Constructing Graph (SCG) module that learns a long-range dependency graph
directly from the image and uses it to propagate contextual information
efficiently to improve semantic segmentation. The module is optimised via a
novel adaptive diagonal enhancement method and a variational lower bound that
consists of a customized graph reconstruction term and a Kullback-Leibler
divergence regularization term. When incorporated into a neural network
(SCG-Net), semantic segmentation is performed in an end-to-end manner and
competitive performance (mean F1-scores of 92.0% and 89.8% respectively) on the
publicly available ISPRS Potsdam and Vaihingen datasets is achieved, with much
fewer parameters, and at a lower computational cost compared to related pure
convolutional neural network (CNN) based models.
