  Explainable AI (XAI) has revolutionized the field of deep learning by
empowering users to have more trust in neural network models. The field of XAI
allows users to probe the inner workings of these algorithms to elucidate their
decision-making processes. The rise in popularity of XAI has led to the advent
of different strategies to produce explanations, all of which only occasionally
agree. Thus several objective evaluation metrics have been devised to decide
which of these modules give the best explanation for specific scenarios. The
goal of the paper is twofold: (i) we employ the notions of necessity and
sufficiency from causal literature to come up with a novel explanatory
technique called SHifted Adversaries using Pixel Elimination(SHAPE) which
satisfies all the theoretical and mathematical criteria of being a valid
explanation, (ii) we show that SHAPE is, infact, an adversarial explanation
that fools causal metrics that are employed to measure the robustness and
reliability of popular importance based visual XAI methods. Our analysis shows
that SHAPE outperforms popular explanatory techniques like GradCAM and
GradCAM++ in these tests and is comparable to RISE, raising questions about the
sanity of these metrics and the need for human involvement for an overall
better evaluation.
