  Multi-modality data is becoming readily available in remote sensing (RS) and
can provide complementary information about the Earth's surface. Effective
fusion of multi-modal information is thus important for various applications in
RS, but also very challenging due to large domain differences, noise, and
redundancies. There is a lack of effective and scalable fusion techniques for
bridging multiple modality encoders and fully exploiting complementary
information. To this end, we propose a new multi-modality network (MultiModNet)
for land cover mapping of multi-modal remote sensing data based on a novel
pyramid attention fusion (PAF) module and a gated fusion unit (GFU). The PAF
module is designed to efficiently obtain rich fine-grained contextual
representations from each modality with a built-in cross-level and cross-view
attention fusion mechanism, and the GFU module utilizes a novel gating
mechanism for early merging of features, thereby diminishing hidden
redundancies and noise. This enables supplementary modalities to effectively
extract the most valuable and complementary information for late feature
fusion. Extensive experiments on two representative RS benchmark datasets
demonstrate the effectiveness, robustness, and superiority of the MultiModNet
for multi-modal land cover classification.
