  Graph convolutional networks (GCNs) have been proved to be very practical to
handle various graph-related tasks. It has attracted considerable research
interest to study deep GCNs, due to their potential superior performance
compared with shallow ones. However, simply increasing network depth will, on
the contrary, hurt the performance due to the over-smoothing problem. Adding
residual connection is proved to be effective for learning deep convolutional
neural networks (deep CNNs), it is not trivial when applied to deep GCNs.
Recent works proposed an initial residual mechanism that did alleviate the
over-smoothing problem in deep GCNs. However, according to our study, their
algorithms are quite sensitive to different datasets. In their setting, the
personalization (dynamic) and correlation (evolving) of how residual applies
are ignored. To this end, we propose a novel model called Dynamic evolving
initial Residual Graph Convolutional Network (DRGCN). Firstly, we use a dynamic
block for each node to adaptively fetch information from the initial
representation. Secondly, we use an evolving block to model the residual
evolving pattern between layers. Our experimental results show that our model
effectively relieves the problem of over-smoothing in deep GCNs and outperforms
the state-of-the-art (SOTA) methods on various benchmark datasets. Moreover, we
develop a mini-batch version of DRGCN which can be applied to large-scale data.
Coupling with several fair training techniques, our model reaches new SOTA
results on the large-scale ogbn-arxiv dataset of Open Graph Benchmark (OGB).
Our reproducible code is available on GitHub.
