  Deep neural networks have shown their profound impact on achieving human
level performance in visual saliency prediction. However, it is still unclear
how they learn the task and what it means in terms of understanding human
visual system. In this work, we develop a technique to derive explainable
saliency models from their corresponding deep neural architecture based
saliency models by applying human perception theories and the conventional
concepts of saliency. This technique helps us understand the learning pattern
of the deep network at its intermediate layers through their activation maps.
Initially, we consider two state-of-the-art deep saliency models, namely UNISAL
and MSI-Net for our interpretation. We use a set of biologically plausible
log-gabor filters for identifying and reconstructing the activation maps of
them using our explainable saliency model. The final saliency map is generated
using these reconstructed activation maps. We also build our own deep saliency
model named cross-concatenated multi-scale residual block based network
(CMRNet) for saliency prediction. Then, we evaluate and compare the performance
of the explainable models derived from UNISAL, MSI-Net and CMRNet on three
benchmark datasets with other state-of-the-art methods. Hence, we propose that
this approach of explainability can be applied to any deep visual saliency
model for interpretation which makes it a generic one.
