  The development of accurate methods for multi-label classification (MLC) of
remote sensing (RS) images is one of the most important research topics in RS.
To address MLC problems, the use of deep neural networks that require a high
number of reliable training images annotated by multiple land-cover class
labels (multi-labels) has been found popular in RS. However, collecting such
annotations is time-consuming and costly. A common procedure to obtain
annotations at zero labeling cost is to rely on thematic products or
crowdsourced labels. As a drawback, these procedures come with the risk of
label noise that can distort the learning process of the MLC algorithms. In the
literature, most label noise robust methods are designed for single-label
classification (SLC) problems in computer vision (CV), where each image is
annotated by a single label. Unlike SLC, label noise in MLC can be associated
with: 1) subtractive label-noise (a land cover class label is not assigned to
an image while that class is present in the image); 2) additive label-noise (a
land cover class label is assigned to an image although that class is not
present in the given image); and 3) mixed label-noise (a combination of both).
In this paper, we investigate three different noise robust CV SLC methods and
adapt them to be robust for multi-label noise scenarios in RS. During
experiments, we study the effects of different types of multi-label noise and
evaluate the adapted methods rigorously. To this end, we also introduce a
synthetic multi-label noise injection strategy that is more adequate to
simulate operational scenarios compared to the uniform label noise injection
strategy, in which the labels of absent and present classes are flipped at
uniform probability. Further, we study the relevance of different evaluation
metrics in MLC problems under noisy multi-labels.
