  Part-prototype networks (e.g., ProtoPNet, ProtoTree, and ProtoPool) have
attracted broad research interest for their intrinsic interpretability and
comparable accuracy to non-interpretable counterparts. However, recent works
find that the interpretability from prototypes is fragile, due to the semantic
gap between the similarities in the feature space and that in the input space.
In this work, we strive to address this challenge by making the first attempt
to quantitatively and objectively evaluate the interpretability of the
part-prototype networks. Specifically, we propose two evaluation metrics,
termed as consistency score and stability score, to evaluate the explanation
consistency across images and the explanation robustness against perturbations,
respectively, both of which are essential for explanations taken into practice.
Furthermore, we propose an elaborated part-prototype network with a
shallow-deep feature alignment (SDFA) module and a score aggregation (SA)
module to improve the interpretability of prototypes. We conduct systematical
evaluation experiments and provide substantial discussions to uncover the
interpretability of existing part-prototype networks. Experiments on three
benchmarks across nine architectures demonstrate that our model achieves
significantly superior performance to the state of the art, in both the
accuracy and interpretability. Our code is available at
https://github.com/hqhQAQ/EvalProtoPNet.
