  Counterfactual explanations have emerged as a popular solution for the
eXplainable AI (XAI) problem of elucidating the predictions of black-box
deep-learning systems due to their psychological validity, flexibility across
problem domains and proposed legal compliance. While over 100 counterfactual
methods exist, claiming to generate plausible explanations akin to those
preferred by people, few have actually been tested on users ($\sim7\%$). So,
the psychological validity of these counterfactual algorithms for effective XAI
for image data is not established. This issue is addressed here using a novel
methodology that (i) gathers ground truth human-generated counterfactual
explanations for misclassified images, in two user studies and, then, (ii)
compares these human-generated ground-truth explanations to
computationally-generated explanations for the same misclassifications. Results
indicate that humans do not "minimally edit" images when generating
counterfactual explanations. Instead, they make larger, "meaningful" edits that
better approximate prototypes in the counterfactual class.
