  Large pre-trained language models have achieved impressive results on various
style classification tasks, but they often learn spurious domain-specific words
to make predictions (Hayati et al., 2021). While human explanation highlights
stylistic tokens as important features for this task, we observe that model
explanations often do not align with them. To tackle this issue, we introduce
StyLEx, a model that learns from human-annotated explanations of stylistic
features and jointly learns to perform the task and predict these features as
model explanations. Our experiments show that StyLEx can provide human-like
stylistic lexical explanations without sacrificing the performance of
sentence-level style prediction on both in-domain and out-of-domain datasets.
Explanations from StyLEx show significant improvements in explanation metrics
(sufficiency, plausibility) and when evaluated with human annotations. They are
also more understandable by human judges compared to the widely-used
saliency-based explanation baseline.
