  To collect large scale annotated data, it is inevitable to introduce label
noise, i.e., incorrect class labels. To be robust against label noise, many
successful methods rely on the noisy classifiers (i.e., models trained on the
noisy training data) to determine whether a label is trustworthy. However, it
remains unknown why this heuristic works well in practice. In this paper, we
provide the first theoretical explanation for these methods. We prove that the
prediction of a noisy classifier can indeed be a good indicator of whether the
label of a training data is clean. Based on the theoretical result, we propose
a novel algorithm that corrects the labels based on the noisy classifier
prediction. The corrected labels are consistent with the true Bayesian optimal
classifier with high probability. We incorporate our label correction algorithm
into the training of deep neural networks and train models that achieve
superior testing performance on multiple public datasets.
