  Deep learning is dramatically transforming the field of medical imaging and
radiology, enabling the identification of pathologies in medical images,
including computed tomography (CT) and X-ray scans. However, the performance of
deep learning models, particularly in segmentation tasks, is often limited by
the need for extensive annotated datasets. To address this challenge, the
capabilities of weakly supervised semantic segmentation are explored through
the lens of Explainable AI and the generation of counterfactual explanations.
The scope of this research is development of a novel counterfactual inpainting
approach (COIN) that flips the predicted classification label from abnormal to
normal by using a generative model. For instance, if the classifier deems an
input medical image X as abnormal, indicating the presence of a pathology, the
generative model aims to inpaint the abnormal region, thus reversing the
classifier's original prediction label. The approach enables us to produce
precise segmentations for pathologies without depending on pre-existing
segmentation masks. Crucially, image-level labels are utilized, which are
substantially easier to acquire than creating detailed segmentation masks. The
effectiveness of the method is demonstrated by segmenting synthetic targets and
actual kidney tumors from CT images acquired from Tartu University Hospital in
Estonia. The findings indicate that COIN greatly surpasses established
attribution methods, such as RISE, ScoreCAM, and LayerCAM, as well as an
alternative counterfactual explanation method introduced by Singla et al. This
evidence suggests that COIN is a promising approach for semantic segmentation
of tumors in CT images, and presents a step forward in making deep learning
applications more accessible and effective in healthcare, where annotated data
is scarce.
