  The problem of open-set noisy labels denotes that part of training data have
a different label space that does not contain the true class. Lots of
approaches, e.g., loss correction and label correction, cannot handle such
open-set noisy labels well, since they need training data and test data to
share the same label space, which does not hold for learning with open-set
noisy labels. The state-of-the-art methods thus employ the sample selection
approach to handle open-set noisy labels, which tries to select clean data from
noisy data for network parameters updates. The discarded data are seen to be
mislabeled and do not participate in training. Such an approach is intuitive
and reasonable at first glance. However, a natural question could be raised
"can such data only be discarded during training?". In this paper, we show that
the answer is no. Specifically, we discuss that the instances of discarded data
could consist of some meaningful information for generalization. For this
reason, we do not abandon such data, but use instance correction to modify the
instances of the discarded data, which makes the predictions for the discarded
data consistent with given labels. Instance correction are performed by
targeted adversarial attacks. The corrected data are then exploited for
training to help generalization. In addition to the analytical results, a
series of empirical evidences are provided to justify our claims.
