  Recently deep neural networks have shown their capacity to memorize training
data, even with noisy labels, which hurts generalization performance. To
mitigate this issue, we provide a simple but effective baseline method that is
robust to noisy labels, even with severe noise. Our objective involves a
variance regularization term that implicitly penalizes the Jacobian norm of the
neural network on the whole training set (including the noisy-labeled data),
which encourages generalization and prevents overfitting to the corrupted
labels. Experiments on both synthetically generated incorrect labels and
realistic large-scale noisy datasets demonstrate that our approach achieves
state-of-the-art performance with a high tolerance to severe noise.
