  Heterogeneous Graph Neural Networks (HGNNs) are a class of powerful deep
learning methods widely used to learn representations of heterogeneous graphs.
Despite the fast development of HGNNs, they still face some challenges such as
over-smoothing, and non-robustness. Previous studies have shown that these
problems can be reduced by using gradient regularization methods. However, the
existing gradient regularization methods focus on either graph topology or node
features. There is no universal approach to integrate these features, which
severely affects the efficiency of regularization. In addition, the inclusion
of gradient regularization into HGNNs sometimes leads to some problems, such as
an unstable training process, increased complexity and insufficient coverage
regularized information. Furthermore, there is still short of a complete
theoretical analysis of the effects of gradient regularization on HGNNs. In
this paper, we propose a novel gradient regularization method called Grug,
which iteratively applies regularization to the gradients generated by both
propagated messages and the node features during the message-passing process.
Grug provides a unified framework integrating graph topology and node features,
based on which we conduct a detailed theoretical analysis of their
effectiveness. Specifically, the theoretical analyses elaborate the advantages
of Grug: 1) Decreasing sample variance during the training process (Stability);
2) Enhancing the generalization of the model (Universality); 3) Reducing the
complexity of the model (Simplicity); 4) Improving the integrity and diversity
of graph information utilization (Diversity). As a result, Grug has the
potential to surpass the theoretical upper bounds set by DropMessage (AAAI-23
Distinguished Papers). In addition, we evaluate Grug on five public real-world
datasets with two downstream tasks...
