  In this paper, we propose an efficient and effective framework to fuse
hyperspectral and Light Detection And Ranging (LiDAR) data using two coupled
convolutional neural networks (CNNs). One CNN is designed to learn
spectral-spatial features from hyperspectral data, and the other one is used to
capture the elevation information from LiDAR data. Both of them consist of
three convolutional layers, and the last two convolutional layers are coupled
together via a parameter sharing strategy. In the fusion phase, feature-level
and decision-level fusion methods are simultaneously used to integrate these
heterogeneous features sufficiently. For the feature-level fusion, three
different fusion strategies are evaluated, including the concatenation
strategy, the maximization strategy, and the summation strategy. For the
decision-level fusion, a weighted summation strategy is adopted, where the
weights are determined by the classification accuracy of each output. The
proposed model is evaluated on an urban data set acquired over Houston, USA,
and a rural one captured over Trento, Italy. On the Houston data, our model can
achieve a new record overall accuracy of 96.03%. On the Trento data, it
achieves an overall accuracy of 99.12%. These results sufficiently certify the
effectiveness of our proposed model.
