  Two main approaches for evaluating the quality of machine-generated
rationales are: 1) using human rationales as a gold standard; and 2) automated
metrics based on how rationales affect model behavior. An open question,
however, is how human rationales fare with these automatic metrics. Analyzing a
variety of datasets and models, we find that human rationales do not
necessarily perform well on these metrics. To unpack this finding, we propose
improved metrics to account for model-dependent baseline performance. We then
propose two methods to further characterize rationale quality, one based on
model retraining and one on using "fidelity curves" to reveal properties such
as irrelevance and redundancy. Our work leads to actionable suggestions for
evaluating and characterizing rationales.
