  Graph neural networks (GNNs) are the dominant paradigm for classifying nodes
in a graph, but they have several undesirable attributes stemming from their
message passing architecture. Recently, distillation methods succeeded in
eliminating the use of GNNs at test time but they still require them during
training. We perform a careful analysis of the role that GNNs play in
distillation methods. This analysis leads us to propose a fully GNN-free
approach for node classification, not requiring them at train or test time. Our
method consists of three key components: smoothness constraints,
pseudo-labeling iterations and neighborhood-label histograms. Our final
approach can match the state-of-the-art accuracy on standard popular benchmarks
such as citation and co-purchase networks, without training a GNN.
