  Attribution methods compute importance scores for input features to explain
the output predictions of deep models. However, accurate assessment of
attribution methods is challenged by the lack of benchmark fidelity for
attributing model predictions. Moreover, other confounding factors in
attribution estimation, including the setup choices of post-processing
techniques and explained model predictions, further compromise the reliability
of the evaluation. In this work, we first identify a set of fidelity criteria
that reliable benchmarks for attribution methods are expected to fulfill,
thereby facilitating a systematic assessment of attribution benchmarks. Next,
we introduce a Backdoor-based eXplainable AI benchmark (BackX) that adheres to
the desired fidelity criteria. We theoretically establish the superiority of
our approach over the existing benchmarks for well-founded attribution
evaluation. With extensive analysis, we also identify a setup for a consistent
and fair benchmarking of attribution methods across different underlying
methodologies. This setup is ultimately employed for a comprehensive comparison
of existing methods using our BackX benchmark. Finally, our analysis also
provides guidance for defending against backdoor attacks with the help of
attribution methods.
