  To better understand scene images in the field of remote sensing, multi-label
annotation of scene images is necessary. Moreover, to enhance the performance
of deep learning models for dealing with semantic scene understanding tasks, it
is vital to train them on large-scale annotated data. However, most existing
datasets are annotated by a single label, which cannot describe the complex
remote sensing images well because scene images might have multiple land cover
classes. Few multi-label high spatial resolution remote sensing datasets have
been developed to train deep learning models for multi-label based tasks, such
as scene classification and image retrieval. To address this issue, in this
paper, we construct a multi-label high spatial resolution remote sensing
dataset named MLRSNet for semantic scene understanding with deep learning from
the overhead perspective. It is composed of high-resolution optical satellite
or aerial images. MLRSNet contains a total of 109,161 samples within 46 scene
categories, and each image has at least one of 60 predefined labels. We have
designed visual recognition tasks, including multi-label based image
classification and image retrieval, in which a wide variety of deep learning
approaches are evaluated with MLRSNet. The experimental results demonstrate
that MLRSNet is a significant benchmark for future research, and it complements
the current widely used datasets such as ImageNet, which fills gaps in
multi-label image research. Furthermore, we will continue to expand the
MLRSNet. MLRSNet and all related materials have been made publicly available at
https://data.mendeley.com/datasets/7j9bv9vwsx/2 and
https://github.com/cugbrs/MLRSNet.git.
