  In this paper, we introduce Planet-CR, a benchmark dataset for
high-resolution cloud removal with multi-modal and multi-resolution data
fusion. Planet-CR is the first public dataset for cloud removal to feature
globally sampled high resolution optical observations, in combination with
paired radar measurements as well as pixel-level land cover annotations. It
provides solid basis for exhaustive evaluation in terms of generating visually
pleasing textures and semantically meaningful structures. With this dataset, we
consider the problem of cloud removal in high resolution optical remote sensing
imagery by integrating multi-modal and multi-resolution information. Existing
multi-modal data fusion based methods, which assume the image pairs are aligned
pixel-to-pixel, are hence not appropriate for this problem. To this end, we
design a new baseline named Align-CR to perform the low-resolution SAR image
guided high-resolution optical image cloud removal. It implicitly aligns the
multi-modal and multi-resolution data during the reconstruction process to
promote the cloud removal performance. The experimental results demonstrate
that the proposed Align-CR method gives the best performance in both visual
recovery quality and semantic recovery quality. The project is available at
https://github.com/zhu-xlab/Planet-CR, and hope this will inspire future
research.
