  Deep learning (DL) has revolutionized the field of document image analysis,
showcasing superhuman performance across a diverse set of tasks. However, the
inherent black-box nature of deep learning models still presents a significant
challenge to their safe and robust deployment in industry. Regrettably, while a
plethora of research has been dedicated in recent years to the development of
DL-powered document analysis systems, research addressing their transparency
aspects has been relatively scarce. In this paper, we aim to bridge this
research gap by introducing DocXplain, a novel model-agnostic explainability
method specifically designed for generating high interpretability feature
attribution maps for the task of document image classification. In particular,
our approach involves independently segmenting the foreground and background
features of the documents into different document elements and then ablating
these elements to assign feature importance. We extensively evaluate our
proposed approach in the context of document image classification, utilizing 4
different evaluation metrics, 2 widely recognized document benchmark datasets,
and 10 state-of-the-art document image classification models. By conducting a
thorough quantitative and qualitative analysis against 9 existing
state-of-the-art attribution methods, we demonstrate the superiority of our
approach in terms of both faithfulness and interpretability. To the best of the
authors' knowledge, this work presents the first model-agnostic
attribution-based explainability method specifically tailored for document
images. We anticipate that our work will significantly contribute to advancing
research on transparency, fairness, and robustness of document image
classification models.
