  With the availability of large datasets and ever-increasing computing power,
there has been a growing use of data-driven artificial intelligence systems,
which have shown their potential for successful application in diverse areas.
However, many of these systems are not able to provide information about the
rationale behind their decisions to their users. Lack of understanding of such
decisions can be a major drawback, especially in critical domains such as those
related to cybersecurity. In light of this problem, in this paper we make three
contributions: (i) proposal and discussion of desiderata for the explanation of
outputs generated by AI-based cybersecurity systems; (ii) a comparative
analysis of approaches in the literature on Explainable Artificial Intelligence
(XAI) under the lens of both our desiderata and further dimensions that are
typically used for examining XAI approaches; and (iii) a general architecture
that can serve as a roadmap for guiding research efforts towards the
development of explainable AI-based cybersecurity systems -- at its core, this
roadmap proposes combinations of several research lines in a novel way towards
tackling the unique challenges that arise in this context.
