  Contrastive explanations, where one decision is explained in contrast to
another, are supposed to be closer to how humans explain a decision than
non-contrastive explanations, where the decision is not necessarily referenced
to an alternative. This claim has never been empirically validated. We analyze
four English text-classification datasets (SST2, DynaSent, BIOS and
DBpedia-Animals). We fine-tune and extract explanations from three different
models (RoBERTa, GTP-2, and T5), each in three different sizes and apply three
post-hoc explainability methods (LRP, GradientxInput, GradNorm). We furthermore
collect and release human rationale annotations for a subset of 100 samples
from the BIOS dataset for contrastive and non-contrastive settings. A
cross-comparison between model-based rationales and human annotations, both in
contrastive and non-contrastive settings, yields a high agreement between the
two settings for models as well as for humans. Moreover, model-based
explanations computed in both settings align equally well with human
rationales. Thus, we empirically find that humans do not necessarily explain in
a contrastive manner.9 pages, long paper at ACL 2022 proceedings.
