  Graph neural networks (GNNs) integrate deep architectures and topological
structure modeling in an effective way. However, the performance of existing
GNNs would decrease significantly when they stack many layers, because of the
over-smoothing issue. Node embeddings tend to converge to similar vectors when
GNNs keep recursively aggregating the representations of neighbors. To enable
deep GNNs, several methods have been explored recently. But they are developed
from either techniques in convolutional neural networks or heuristic
strategies. There is no generalizable and theoretical principle to guide the
design of deep GNNs. To this end, we analyze the bottleneck of deep GNNs by
leveraging the Dirichlet energy of node embeddings, and propose a generalizable
principle to guide the training of deep GNNs. Based on it, a novel deep GNN
framework -- EGNN is designed. It could provide lower and upper constraints in
terms of Dirichlet energy at each layer to avoid over-smoothing. Experimental
results demonstrate that EGNN achieves state-of-the-art performance by using
deep layers.
