  Despite the state-of-the-art performance of deep convolutional neural
networks, they are susceptible to bias and malfunction in unseen situations.
Moreover, the complex computation behind their reasoning is not
human-understandable to develop trust. External explainer methods have tried to
interpret network decisions in a human-understandable way, but they are accused
of fallacies due to their assumptions and simplifications. On the other side,
the inherent self-interpretability of models, while being more robust to the
mentioned fallacies, cannot be applied to the already trained models. In this
work, we propose a new attention-based pooling layer, called Local Attention
Pooling (LAP), that accomplishes self-interpretability and the possibility for
knowledge injection without performance loss. The module is easily pluggable
into any convolutional neural network, even the already trained ones. We have
defined a weakly supervised training scheme to learn the distinguishing
features in decision-making without depending on experts' annotations. We
verified our claims by evaluating several LAP-extended models on two datasets,
including ImageNet. The proposed framework offers more valid
human-understandable and faithful-to-the-model interpretations than the
commonly used white-box explainer methods.
