  Temporal Graph Neural Network (TGNN) has been receiving a lot of attention
recently due to its capability in modeling time-evolving graph-related tasks.
Similar to Graph Neural Networks, it is also non-trivial to interpret
predictions made by a TGNN due to its black-box nature. A major approach
tackling this problems in GNNs is by analyzing the model' responses on some
perturbations of the model's inputs, called perturbation-based explanation
methods. While these methods are convenient and flexible since they do not need
internal access to the model, does this lack of internal access prevent them
from revealing some important information of the predictions? Motivated by that
question, this work studies the limit of some classes of perturbation-based
explanation methods. Particularly, by constructing some specific instances of
TGNNs, we show (i) node-perturbation cannot reliably identify the paths
carrying out the prediction, (ii) edge-perturbation is not reliable in
determining all nodes contributing to the prediction and (iii) perturbing both
nodes and edges does not reliably help us identify the graph's components
carrying out the temporal aggregation in TGNNs.
