  While much research focused on producing explanations, it is still unclear
how the produced explanations' quality can be evaluated in a meaningful way.
Today's predominant approach is to quantify explanations using proxy scores
which compare explanations to (human-annotated) gold explanations. This
approach assumes that explanations which reach higher proxy scores will also
provide a greater benefit to human users. In this paper, we present problems of
this approach. Concretely, we (i) formulate desired characteristics of
explanation quality, (ii) describe how current evaluation practices violate
them, and (iii) support our argumentation with initial evidence from a
crowdsourcing case study in which we investigate the explanation quality of
state-of-the-art explainable question answering systems. We find that proxy
scores correlate poorly with human quality ratings and, additionally, become
less expressive the more often they are used (i.e. following Goodhart's law).
Finally, we propose guidelines to enable a meaningful evaluation of
explanations to drive the development of systems that provide tangible benefits
to human users.
