  As remote sensing (RS) data obtained from different sensors become available
largely and openly, multimodal data processing and analysis techniques have
been garnering increasing interest in the RS and geoscience community. However,
due to the gap between different modalities in terms of imaging sensors,
resolutions, and contents, embedding their complementary information into a
consistent, compact, accurate, and discriminative representation, to a great
extent, remains challenging. To this end, we propose a shared and specific
feature learning (S2FL) model. S2FL is capable of decomposing multimodal RS
data into modality-shared and modality-specific components, enabling the
information blending of multi-modalities more effectively, particularly for
heterogeneous data sources. Moreover, to better assess multimodal baselines and
the newly-proposed S2FL model, three multimodal RS benchmark datasets, i.e.,
Houston2013 -- hyperspectral and multispectral data, Berlin -- hyperspectral
and synthetic aperture radar (SAR) data, Augsburg -- hyperspectral, SAR, and
digital surface model (DSM) data, are released and used for land cover
classification. Extensive experiments conducted on the three datasets
demonstrate the superiority and advancement of our S2FL model in the task of
land cover classification in comparison with previously-proposed
state-of-the-art baselines. Furthermore, the baseline codes and datasets used
in this paper will be made available freely at
https://github.com/danfenghong/ISPRS_S2FL.
