  Existing multiple modality fusion methods, such as concatenation, summation,
and encoder-decoder-based fusion, have recently been employed to combine
modality characteristics of Hyperspectral Image (HSI) and Light Detection And
Ranging (LiDAR). However, these methods consider the relationship of HSI-LiDAR
signals from limited perspectives. More specifically, they overlook the
contextual information across modalities of HSI and LiDAR and the
intra-modality characteristics of LiDAR. In this paper, we provide a new
insight into feature fusion to explore the relationships across HSI and LiDAR
modalities comprehensively. An Interconnected Fusion (IF) framework is
proposed. Firstly, the center patch of the HSI input is extracted and
replicated to the size of the HSI input. Then, nine different perspectives in
the fusion matrix are generated by calculating self-attention and
cross-attention among the replicated center patch, HSI input, and corresponding
LiDAR input. In this way, the intra- and inter-modality characteristics can be
fully exploited, and contextual information is considered in both
intra-modality and inter-modality manner. These nine interrelated elements in
the fusion matrix can complement each other and eliminate biases, which can
generate a multi-modality representation for classification accurately.
Extensive experiments have been conducted on three widely used datasets:
Trento, MUUFL, and Houston. The IF framework achieves state-of-the-art results
on these datasets compared to existing approaches.
