  Multi-label image classification has generated significant interest in recent
years and the performance of such systems often suffers from the not so
infrequent occurrence of incorrect or missing labels in the training data. In
this paper, we extend the state-of the-art of training classifiers to jointly
deal with both forms of errorful data. We accomplish this by modeling noisy and
missing labels in multi-label images with a new Noise Modeling Network (NMN)
that follows our convolutional neural network (CNN), integrates with it,
forming an end-to-end deep learning system, which can jointly learn the noise
distribution and CNN parameters. The NMN learns the distribution of noise
patterns directly from the noisy data without the need for any clean training
data. The NMN can model label noise that depends only on the true label or is
also dependent on the image features. We show that the integrated NMN/CNN
learning system consistently improves the classification performance, for
different levels of label noise, on the MSR-COCO dataset and MSR-VTT dataset.
We also show that noise performance improvements are obtained when multiple
instance learning methods are used.
