  Noisy labels are very common in real-world training data, which lead to poor
generalization on test data because of overfitting to the noisy labels. In this
paper, we claim that such overfitting can be avoided by "early stopping"
training a deep neural network before the noisy labels are severely memorized.
Then, we resume training the early stopped network using a "maximal safe set,"
which maintains a collection of almost certainly true-labeled samples at each
epoch since the early stop point. Putting them all together, our novel
two-phase training method, called Prestopping, realizes noise-free training
under any type of label noise for practical use. Extensive experiments using
four image benchmark data sets verify that our method significantly outperforms
four state-of-the-art methods in test error by 0.4-8.2 percent points under
existence of real-world noise.
