  Recent work on explainable NLP has shown that few-shot prompting can enable
large pretrained language models (LLMs) to generate grammatical and factual
natural language explanations for data labels. In this work, we study the
connection between explainability and sample hardness by investigating the
following research question - "Are LLMs and humans equally good at explaining
data labels for both easy and hard samples?" We answer this question by first
collecting human-written explanations in the form of generalizable commonsense
rules on the task of Winograd Schema Challenge (Winogrande dataset). We compare
these explanations with those generated by GPT-3 while varying the hardness of
the test samples as well as the in-context samples. We observe that (1) GPT-3
explanations are as grammatical as human explanations regardless of the
hardness of the test samples, (2) for easy examples, GPT-3 generates highly
supportive explanations but human explanations are more generalizable, and (3)
for hard examples, human explanations are significantly better than GPT-3
explanations both in terms of label-supportiveness and generalizability
judgements. We also find that hardness of the in-context examples impacts the
quality of GPT-3 explanations. Finally, we show that the supportiveness and
generalizability aspects of human explanations are also impacted by sample
hardness, although by a much smaller margin than models. Supporting code and
data are available at https://github.com/swarnaHub/ExplanationHardness
