  Remote sensing image retrieval(RSIR), which aims to efficiently retrieve data
of interest from large collections of remote sensing data, is a fundamental
task in remote sensing. Over the past several decades, there has been
significant effort to extract powerful feature representations for this task
since the retrieval performance depends on the representative strength of the
features. Benchmark datasets are also critical for developing, evaluating, and
comparing RSIR approaches. Current benchmark datasets are deficient in that 1)
they were originally collected for land use/land cover classification and not
image retrieval, 2) they are relatively small in terms of the number of classes
as well the number of sample images per class, and 3) the retrieval performance
has saturated. These limitations have severely restricted the development of
novel feature representations for RSIR, particularly the recent deep-learning
based features which require large amounts of training data. We therefore
present in this paper, a new large-scale remote sensing dataset termed
"PatternNet" that was collected specifically for RSIR. PatternNet was collected
from high-resolution imagery and contains 38 classes with 800 images per class.
We also provide a thorough review of RSIR approaches ranging from traditional
handcrafted feature based methods to recent deep learning based ones. We
evaluate over 35 methods to establish extensive baseline results for future
RSIR research using the PatternNet benchmark.
