  Mislabeled examples are a common issue in real-world data, particularly for
tasks like token classification where many labels must be chosen on a
fine-grained basis. Here we consider the task of finding sentences that contain
label errors in token classification datasets. We study 11 different
straightforward methods that score tokens/sentences based on the predicted
class probabilities output by a (any) token classification model (trained via
any procedure). In precision-recall evaluations based on real-world label
errors in entity recognition data from CoNLL-2003, we identify a simple and
effective method that consistently detects those sentences containing label
errors when applied with different token classification models.
