  Despite their high accuracies, modern complex image classifiers cannot be
trusted for sensitive tasks due to their unknown decision-making process and
potential biases. Counterfactual explanations are very effective in providing
transparency for these black-box algorithms. Nevertheless, generating
counterfactuals that can have a consistent impact on classifier outputs and yet
expose interpretable feature changes is a very challenging task. We introduce a
novel method to generate causal and yet interpretable counterfactual
explanations for image classifiers using pretrained generative models without
any re-training or conditioning. The generative models in this technique are
not bound to be trained on the same data as the target classifier. We use this
framework to obtain contrastive and causal sufficiency and necessity scores as
global explanations for black-box classifiers. On the task of face attribute
classification, we show how different attributes influence the classifier
output by providing both causal and contrastive feature attributions, and the
corresponding counterfactual images.
