  The performance of convolutional neural networks has continued to improve
over the last decade. At the same time, as model complexity grows, it becomes
increasingly more difficult to explain model decisions. Such explanations may
be of critical importance for reliable operation of human-machine pairing
setups, or for model selection when the "best" model among many
equally-accurate models must be established. Saliency maps represent one
popular way of explaining model decisions by highlighting image regions models
deem important when making a prediction. However, examining salience maps at
scale is not practical. In this paper, we propose five novel methods of
leveraging model salience to explain a model behavior at scale. These methods
ask: (a) what is the average entropy for a model's salience maps, (b) how does
model salience change when fed out-of-set samples, (c) how closely does model
salience follow geometrical transformations, (d) what is the stability of model
salience across independent training runs, and (e) how does model salience
react to salience-guided image degradations. To assess the proposed measures on
a concrete and topical problem, we conducted a series of experiments for the
task of synthetic face detection with two types of models: those trained
traditionally with cross-entropy loss, and those guided by human salience when
training to increase model generalizability. These two types of models are
characterized by different, interpretable properties of their salience maps,
which allows for the evaluation of the correctness of the proposed measures. We
offer source codes for each measure along with this paper.
