  Image attribution algorithms aim to identify important regions that are
highly relevant to model decisions. Although existing attribution solutions can
effectively assign importance to target elements, they still face the following
challenges: 1) existing attribution methods generate inaccurate small regions
thus misleading the direction of correct attribution, and 2) the model cannot
produce good attribution results for samples with wrong predictions. To address
the above challenges, this paper re-models the above image attribution problem
as a submodular subset selection problem, aiming to enhance model
interpretability using fewer regions. To address the lack of attention to local
regions, we construct a novel submodular function to discover more accurate
small interpretation regions. To enhance the attribution effect for all
samples, we also impose four different constraints on the selection of
sub-regions, i.e., confidence, effectiveness, consistency, and collaboration
scores, to assess the importance of various subsets. Moreover, our theoretical
analysis substantiates that the proposed function is in fact submodular.
Extensive experiments show that the proposed method outperforms SOTA methods on
two face datasets (Celeb-A and VGG-Face2) and one fine-grained dataset
(CUB-200-2011). For correctly predicted samples, the proposed method improves
the Deletion and Insertion scores with an average of 4.9% and 2.5% gain
relative to HSIC-Attribution. For incorrectly predicted samples, our method
achieves gains of 81.0% and 18.4% compared to the HSIC-Attribution algorithm in
the average highest confidence and Insertion score respectively. The code is
released at https://github.com/RuoyuChen10/SMDL-Attribution.
