  Graph Neural Networks (GNNs) are effective in many applications. Still, there
is a limited understanding of the effect of common graph structures on the
learning process of GNNs. In this work, we systematically study the impact of
community structure on the performance of GNNs in semi-supervised node
classification on graphs. Following an ablation study on six datasets, we
measure the performance of GNNs on the original graphs, and the change in
performance in the presence and the absence of community structure. Our results
suggest that communities typically have a major impact on the learning process
and classification performance. For example, in cases where the majority of
nodes from one community share a single classification label, breaking up
community structure results in a significant performance drop. On the other
hand, for cases where labels show low correlation with communities, we find
that the graph structure is rather irrelevant to the learning process, and a
feature-only baseline becomes hard to beat. With our work, we provide deeper
insights in the abilities and limitations of GNNs, including a set of general
guidelines for model selection based on the graph structure.
