  The attribution method provides a direction for interpreting opaque neural
networks in a visual way by identifying and visualizing the input
regions/pixels that dominate the output of a network. Regarding the attribution
method for visually explaining video understanding networks, it is challenging
because of the unique spatiotemporal dependencies existing in video inputs and
the special 3D convolutional or recurrent structures of video understanding
networks. However, most existing attribution methods focus on explaining
networks taking a single image as input and a few works specifically devised
for video attribution come short of dealing with diversified structures of
video understanding networks. In this paper, we investigate a generic
perturbation-based attribution method that is compatible with diversified video
understanding networks. Besides, we propose a novel regularization term to
enhance the method by constraining the smoothness of its attribution results in
both spatial and temporal dimensions. In order to assess the effectiveness of
different video attribution methods without relying on manual judgement, we
introduce reliable objective metrics which are checked by a newly proposed
reliability measurement. We verified the effectiveness of our method by both
subjective and objective evaluation and comparison with multiple significant
attribution methods.
