  We approach the problem of improving robustness of deep learning algorithms
in the presence of label noise. Building upon existing label correction and
co-teaching methods, we propose a novel training procedure to mitigate the
memorization of noisy labels, called CrossSplit, which uses a pair of neural
networks trained on two disjoint parts of the labelled dataset. CrossSplit
combines two main ingredients: (i) Cross-split label correction. The idea is
that, since the model trained on one part of the data cannot memorize
example-label pairs from the other part, the training labels presented to each
network can be smoothly adjusted by using the predictions of its peer network;
(ii) Cross-split semi-supervised training. A network trained on one part of the
data also uses the unlabeled inputs of the other part. Extensive experiments on
CIFAR-10, CIFAR-100, Tiny-ImageNet and mini-WebVision datasets demonstrate that
our method can outperform the current state-of-the-art in a wide range of noise
ratios.
