  Self-supervised learning (SSL) has attracted much interest in remote sensing
and earth observation due to its ability to learn task-agnostic representations
without human annotation. While most of the existing SSL works in remote
sensing utilize ConvNet backbones and focus on a single modality, we explore
the potential of vision transformers (ViTs) for joint SAR-optical
representation learning. Based on DINO, a state-of-the-art SSL algorithm that
distills knowledge from two augmented views of an input image, we combine SAR
and optical imagery by concatenating all channels to a unified input.
Subsequently, we randomly mask out channels of one modality as a data
augmentation strategy. While training, the model gets fed optical-only,
SAR-only, and SAR-optical image pairs learning both inner- and intra-modality
representations. Experimental results employing the BigEarthNet-MM dataset
demonstrate the benefits of both, the ViT backbones and the proposed multimodal
SSL algorithm DINO-MM.
