  Explanation regularisation (ER) has been introduced as a way to guide models
to make their predictions in a manner more akin to humans, i.e., making their
attributions "plausible". This is achieved by introducing an auxiliary
explanation loss, that measures how well the output of an input attribution
technique for the model agrees with relevant human-annotated rationales. One
positive outcome of using ER appears to be improved performance in
out-of-domain (OOD) settings, presumably due to an increased reliance on
"plausible" tokens. However, previous work has under-explored the impact of the
ER objective on model attributions, in particular when obtained with techniques
other than the one used to train ER. In this work, we contribute a study of
ER's effectiveness at informing classification decisions on plausible tokens,
and the relationship between increased plausibility and robustness to OOD
conditions. Through a series of analyses, we find that the connection between
ER and the ability of a classifier to rely on plausible features has been
overstated and that a stronger reliance on plausible tokens does not seem to be
the cause for any perceived OOD improvements.
