  In training machine learning models for land cover semantic segmentation
there is a stark contrast between the availability of satellite imagery to be
used as inputs and ground truth data to enable supervised learning. While
thousands of new satellite images become freely available on a daily basis,
getting ground truth data is still very challenging, time consuming and costly.
In this paper we present Embedding Earth a self-supervised contrastive
pre-training method for leveraging the large availability of satellite imagery
to improve performance on downstream dense land cover classification tasks.
Performing an extensive experimental evaluation spanning four countries and two
continents we use models pre-trained with our proposed method as initialization
points for supervised land cover semantic segmentation and observe significant
improvements up to 25% absolute mIoU. In every case tested we outperform random
initialization, especially so when ground truth data are scarse. Through a
series of ablation studies we explore the qualities of the proposed approach
and find that learnt features can generalize between disparate regions opening
up the possibility of using the proposed pre-training scheme as a replacement
to random initialization for Earth observation tasks. Code will be uploaded
soon at https://github.com/michaeltrs/DeepSatModels.
