  Evaluating explanations of image classifiers regarding ground truth, e.g.
segmentation masks defined by human perception, primarily evaluates the quality
of the models under consideration rather than the explanation methods
themselves. Driven by this observation, we propose a framework for
$\textit{jointly}$ evaluating the robustness of safety-critical systems that
$\textit{combine}$ a deep neural network with an explanation method. These are
increasingly used in real-world applications like medical image analysis or
robotics. We introduce a fine-tuning procedure to (mis)align
model$\unicode{x2013}$explanation pipelines with ground truth and use it to
quantify the potential discrepancy between worst and best-case scenarios of
human alignment. Experiments across various model architectures and post-hoc
local interpretation methods provide insights into the robustness of vision
transformers and the overall vulnerability of such AI systems to potential
adversarial attacks.
