  Learning representations through self-supervision on unlabeled data has
proven highly effective for understanding diverse images. However, remote
sensing images often have complex and densely populated scenes with multiple
land objects and no clear foreground objects. This intrinsic property generates
high object density, resulting in false positive pairs or missing contextual
information in self-supervised learning. To address these problems, we propose
a context-enhanced masked image modeling method (CtxMIM), a simple yet
efficient MIM-based self-supervised learning for remote sensing image
understanding. CtxMIM formulates original image patches as a reconstructive
template and employs a Siamese framework to operate on two sets of image
patches. A context-enhanced generative branch is introduced to provide
contextual information through context consistency constraints in the
reconstruction. With the simple and elegant design, CtxMIM encourages the
pre-training model to learn object-level or pixel-level features on a
large-scale dataset without specific temporal or geographical constraints.
Finally, extensive experiments show that features learned by CtxMIM outperform
fully supervised and state-of-the-art self-supervised learning methods on
various downstream tasks, including land cover classification, semantic
segmentation, object detection, and instance segmentation. These results
demonstrate that CtxMIM learns impressive remote sensing representations with
high generalization and transferability. Code and data will be made public
available.
