  There is broad agreement that Artificial Intelligence (AI) systems,
particularly those using Machine Learning (ML), should be able to "explain"
their behavior. Unfortunately, there is little agreement as to what constitutes
an "explanation." This has caused a disconnect between the explanations that
systems produce in service of explainable Artificial Intelligence (XAI) and
those explanations that users and other audiences actually need, which should
be defined by the full spectrum of functional roles, audiences, and
capabilities for explanation. In this paper, we explore the features of
explanations and how to use those features in evaluating their utility. We
focus on the requirements for explanations defined by their functional role,
the knowledge states of users who are trying to understand them, and the
availability of the information needed to generate them. Further, we discuss
the risk of XAI enabling trust in systems without establishing their
trustworthiness and define a critical next step for the field of XAI to
establish metrics to guide and ground the utility of system-generated
explanations.
