  While a substantial amount of work has recently been devoted to enhance the
performance of computational Authorship Identification (AId) systems, little to
no attention has been paid to endowing AId systems with the ability to explain
the reasons behind their predictions. This lacking substantially hinders the
practical employment of AId methodologies, since the predictions returned by
such systems are hardly useful unless they are supported with suitable
explanations. In this paper, we explore the applicability of existing
general-purpose eXplainable Artificial Intelligence (XAI) techniques to AId,
with a special focus on explanations addressed to scholars working in cultural
heritage. In particular, we assess the relative merits of three different types
of XAI techniques (feature ranking, probing, factuals and counterfactual
selection) on three different AId tasks (authorship attribution, authorship
verification, same-authorship verification) by running experiments on real AId
data. Our analysis shows that, while these techniques make important first
steps towards explainable Authorship Identification, more work remains to be
done in order to provide tools that can be profitably integrated in the
workflows of scholars.
