  Deep neural networks are being increasingly implemented throughout society in
recent years. It is useful to identify which parameters trigger
misclassification in diagnosing undesirable model behaviors. The concept of
parameter saliency is proposed and used to diagnose convolutional neural
networks (CNNs) by ranking convolution filters that may have caused
misclassification on the basis of parameter saliency. It is also shown that
fine-tuning the top ranking salient filters efficiently corrects
misidentification on ImageNet. However, there is still a knowledge gap in terms
of understanding why parameter saliency ranking can find the filters inducing
misidentification. In this work, we attempt to bridge the gap by analyzing
parameter saliency ranking from a statistical viewpoint, namely, extreme value
theory. We first show that the existing work implicitly assumes that the
gradient norm computed for each filter follows a normal distribution. Then, we
clarify the relationship between parameter saliency and the score based on the
peaks-over-threshold (POT) method, which is often used to model extreme values.
Finally, we reformulate parameter saliency in terms of the POT method, where
this reformulation is regarded as statistical anomaly detection and does not
require the implicit assumptions of the existing parameter-saliency
formulation. Our experimental results demonstrate that our reformulation can
detect malicious filters as well. Furthermore, we show that the existing
parameter saliency method exhibits a bias against the depth of layers in deep
neural networks. In particular, this bias has the potential to inhibit the
discovery of filters that cause misidentification in situations where domain
shift occurs. In contrast, parameter saliency based on POT shows less of this
bias.
