  eXplainable Artificial Intelligence (XAI) has emerged as an essential
requirement when dealing with mission-critical applications, ensuring
transparency and interpretability of the employed black box AI models. The
significance of XAI spans various domains, from healthcare to finance, where
understanding the decision-making process of deep learning algorithms is
essential. Most AI-based computer vision models are often black boxes; hence,
providing explainability of deep neural networks in image processing is crucial
for their wide adoption and deployment in medical image analysis, autonomous
driving, and remote sensing applications. Recently, several XAI methods for
image classification tasks have been introduced. On the contrary, image
segmentation has received comparatively less attention in the context of
explainability, although it is a fundamental task in computer vision
applications, especially in remote sensing. Only some research proposes
gradient-based XAI algorithms for image segmentation. This paper adapts the
recent gradient-free Sobol XAI method for semantic segmentation. To measure the
performance of the Sobol method for segmentation, we propose a quantitative XAI
evaluation method based on a learnable noise model. The main objective of this
model is to induce noise on the explanation maps, where higher induced noise
signifies low accuracy and vice versa. A benchmark analysis is conducted to
evaluate and compare performance of three XAI methods, including Seg-Grad-CAM,
Seg-Grad-CAM++ and Seg-Sobol using the proposed noise-based evaluation
technique. This constitutes the first attempt to run and evaluate XAI methods
using high-resolution satellite images.
