  Graph Neural Networks (GNNs) have received much attention in the graph deep
learning domain. However, recent research empirically and theoretically shows
that deep GNNs suffer from over-fitting and over-smoothing problems. The usual
solutions either cannot solve extensive runtime of deep GNNs or restrict graph
convolution in the same feature space. We propose the Adaptive Graph Diffusion
Networks (AGDNs) which perform multi-layer generalized graph diffusion in
different feature spaces with moderate complexity and runtime. Standard graph
diffusion methods combine large and dense powers of the transition matrix with
predefined weighting coefficients. Instead, AGDNs combine smaller multi-hop
node representations with learnable and generalized weighting coefficients. We
propose two scalable mechanisms of weighting coefficients to capture multi-hop
information: Hop-wise Attention (HA) and Hop-wise Convolution (HC). We evaluate
AGDNs on diverse, challenging Open Graph Benchmark (OGB) datasets with
semi-supervised node classification and link prediction tasks. Until the date
of submission (Aug 26, 2022), AGDNs achieve top-1 performance on the
ogbn-arxiv, ogbn-proteins and ogbl-ddi datasets and top-3 performance on the
ogbl-citation2 dataset. On the similar Tesla V100 GPU cards, AGDNs outperform
Reversible GNNs (RevGNNs) with 13% complexity and 1% training runtime of
RevGNNs on the ogbn-proteins dataset. AGDNs also achieve comparable performance
to SEAL with 36% training and 0.2% inference runtime of SEAL on the
ogbl-citation2 dataset.
