  As large-scale, pre-trained language models achieve human-level and
superhuman accuracy on existing language understanding tasks, statistical bias
in benchmark data and probing studies have recently called into question their
true capabilities. For a more informative evaluation than accuracy on text
classification tasks can offer, we propose evaluating systems through a novel
measure of prediction coherence. We apply our framework to two existing
language understanding benchmarks with different properties to demonstrate its
versatility. Our experimental results show that this evaluation framework,
although simple in ideas and implementation, is a quick, effective, and
versatile measure to provide insight into the coherence of machines'
predictions.
