  Graph Convolutional Network (GCN) with the powerful capacity to explore
graph-structural data has gained noticeable success in recent years.
Nonetheless, most of the existing GCN-based models suffer from the notorious
over-smoothing issue, owing to which shallow networks are extensively adopted.
This may be problematic for complex graph datasets because a deeper GCN should
be beneficial to propagating information across remote neighbors. Recent works
have devoted effort to addressing over-smoothing problems, including
establishing residual connection structure or fusing predictions from
multi-layer models. Because of the indistinguishable embeddings from deep
layers, it is reasonable to generate more reliable predictions before
conducting the combination of outputs from various layers. In light of this, we
propose an Alternating Graph-regularized Neural Network (AGNN) composed of
Graph Convolutional Layer (GCL) and Graph Embedding Layer (GEL). GEL is derived
from the graph-regularized optimization containing Laplacian embedding term,
which can alleviate the over-smoothing problem by periodic projection from the
low-order feature space onto the high-order space. With more distinguishable
features of distinct layers, an improved Adaboost strategy is utilized to
aggregate outputs from each layer, which explores integrated embeddings of
multi-hop neighbors. The proposed model is evaluated via a large number of
experiments including performance comparison with some multi-layer or
multi-order graph neural networks, which reveals the superior performance
improvement of AGNN compared with state-of-the-art models.
