  The image-based diagnosis is now a vital aspect of modern automation assisted
diagnosis. To enable models to produce pixel-level diagnosis, pixel-level
ground-truth labels are essentially required. However, since it is often not
straight forward to obtain the labels in many application domains such as in
medical image, classification-based approaches have become the de facto
standard to perform the diagnosis. Though they can identify class-salient
regions, they may not be useful for diagnosis where capturing all of the
evidences is important requirement. Alternatively, a counterfactual explanation
(CX) aims at providing explanations using a casual reasoning process of form
"If X has not happend, Y would not heppend". Existing CX approaches, however,
use classifier to explain features that can change its predictions. Thus, they
can only explain class-salient features, rather than entire object of interest.
This hence motivates us to propose a novel CX strategy that is not reliant on
image classification. This work is inspired from the recent developments in
generative adversarial networks (GANs) based image-to-image domain translation,
and leverages to translate an abnormal image to counterpart normal image (i.e.
counterfactual instance CI) to find discrepancy maps between the two. Since it
is generally not possible to obtain abnormal and normal image pairs, we
leverage Cycle-Consistency principle (a.k.a CycleGAN) to perform the
translation in unsupervised way. We formulate CX in terms of a discrepancy map
that, when added from the abnormal image, will make it indistinguishable from
the CI. We evaluate our method on three datasets including a synthetic,
tuberculosis and BraTS dataset. All these experiments confirm the supremacy of
propose method in generating accurate CX and CI.
