  Deep neural networks, while powerful for image classification, often operate
as "black boxes," complicating the understanding of their decision-making
processes. Various explanation methods, particularly those generating saliency
maps, aim to address this challenge. However, the inconsistency issues of
faithfulness metrics hinder reliable benchmarking of explanation methods. This
paper employs an approach inspired by psychometrics, utilizing Krippendorf's
alpha to quantify the benchmark reliability of post-hoc methods in image
classification. The study proposes model training modifications, including
feeding perturbed samples and employing focal loss, to enhance robustness and
calibration. Empirical evaluations demonstrate significant improvements in
benchmark reliability across metrics, datasets, and post-hoc methods. This
pioneering work establishes a foundation for more reliable evaluation practices
in the realm of post-hoc explanation methods, emphasizing the importance of
model robustness in the assessment process.
