  Graph Neural Networks (GNNs) have shown remarkable effectiveness in capturing
abundant information in graph-structured data. However, the black-box nature of
GNNs hinders users from understanding and trusting the models, thus leading to
difficulties in their applications. While recent years witness the prosperity
of the studies on explaining GNNs, most of them focus on static graphs, leaving
the explanation of dynamic GNNs nearly unexplored. It is challenging to explain
dynamic GNNs, due to their unique characteristic of time-varying graph
structures. Directly using existing models designed for static graphs on
dynamic graphs is not feasible because they ignore temporal dependencies among
the snapshots. In this work, we propose DGExplainer to provide reliable
explanation on dynamic GNNs. DGExplainer redistributes the output activation
score of a dynamic GNN to the relevances of the neurons of its previous layer,
which iterates until the relevance scores of the input neuron are obtained. We
conduct quantitative and qualitative experiments on real-world datasets to
demonstrate the effectiveness of the proposed framework for identifying
important nodes for link prediction and node regression for dynamic GNNs.
