  In the field of fusing multi-spectral and panchromatic images
(Pan-sharpening), the impressive effectiveness of deep neural networks has been
recently employed to overcome the drawbacks of traditional linear models and
boost the fusing accuracy. However, to the best of our knowledge, existing
research works are mainly based on simple and flat networks with relatively
shallow architecture, which severely limited their performances. In this paper,
the concept of residual learning has been introduced to form a very deep
convolutional neural network to make a full use of the high non-linearity of
deep learning models. By both quantitative and visual assessments on a large
number of high quality multi-spectral images from various sources, it has been
supported that our proposed model is superior to all mainstream algorithms
included in the comparison, and achieved the highest spatial-spectral unified
accuracy.
