  While Graph Neural Networks (GNNs) are powerful models for learning
representations on graphs, most state-of-the-art models do not have significant
accuracy gain beyond two to three layers. Deep GNNs fundamentally need to
address: 1). expressivity challenge due to oversmoothing, and 2). computation
challenge due to neighborhood explosion. We propose a simple "deep GNN, shallow
sampler" design principle to improve both the GNN accuracy and efficiency -- to
generate representation of a target node, we use a deep GNN to pass messages
only within a shallow, localized subgraph. A properly sampled subgraph may
exclude irrelevant or even noisy nodes, and still preserve the critical
neighbor features and graph structures. The deep GNN then smooths the
informative local signals to enhance feature learning, rather than
oversmoothing the global graph signals into just "white noise". We
theoretically justify why the combination of deep GNNs with shallow samplers
yields the best learning performance. We then propose various sampling
algorithms and neural architecture extensions to achieve good empirical
results. On the largest public graph dataset, ogbn-papers100M, we achieve
state-of-the-art accuracy with an order of magnitude reduction in hardware
cost.
