  Most explainable AI (XAI) techniques are concerned with the design of
algorithms to explain the AI's decision. However, the data that is used to
train these algorithms may contain features that are often incomprehensible to
an end-user even with the best XAI algorithms. Thus, the problem of
explainability has to be addressed starting right from the data creation step.
In this paper, we studied this problem considering the use-case of explaining
loan denials to end-users as opposed to AI engineers or domain experts.
Motivated by the lack of datasets that are representative of user-friendly
explanations, we build the first-of-its-kind dataset that is representative of
user-friendly explanations for loan denials. The paper shares some of the
insights gained in curating the dataset. First, existing datasets seldom
contain features that end users consider as acceptable in understanding a
model's decision. Second, understanding of the explanation's context such as
the human-in-the-loop seeking the explanation, and the purpose for which an
explanation is sought, aids in the creation of user-friendly datasets. Thus,
our dataset, which we call Xnet, also contains explanations that serve
different purposes: those that educate the loan applicants, and help them take
appropriate action towards a future approval. We hope this work will trigger
the creation of new user friendly datasets, and serve as a guide for the
curation of such datasets.
