  Providing meaningful and actionable explanations to end-users is a
fundamental prerequisite for implementing explainable intelligent systems in
the real world. Explainability is a situated interaction between a user and the
AI system rather than being static design principles. The content of
explanations is context-dependent and must be defined by evidence about the
user and its context. This paper seeks to operationalize this concept by
proposing a formative architecture that defines the explanation space from a
user-inspired perspective. The architecture comprises five intertwined
components to outline explanation requirements for a task: (1) the end-users
mental models, (2) the end-users cognitive process, (3) the user interface, (4)
the human-explainer agent, and the (5) agent process. We first define each
component of the architecture. Then we present the Abstracted Explanation
Space, a modeling tool that aggregates the architecture's components to support
designers in systematically aligning explanations with the end-users work
practices, needs, and goals. It guides the specifications of what needs to be
explained (content - end-users mental model), why this explanation is necessary
(context - end-users cognitive process), to delimit how to explain it (format -
human-explainer agent and user interface), and when should the explanations be
given. We then exemplify the tool's use in an ongoing case study in the
aircraft maintenance domain. Finally, we discuss possible contributions of the
tool, known limitations/areas for improvement, and future work to be done.
