  Pansharpening aims to fuse panchromatic and multispectral images from the
satellite to generate images with both high spatial and spectral resolution.
With the successful applications of deep learning in the computer vision field,
a lot of scholars have proposed many convolutional neural networks (CNNs) to
solve the pansharpening task. These pansharpening networks focused on various
distinctive structures of CNNs, and most of them are trained by L2 loss between
fused images and simulated desired multispectral images. However, L2 loss is
designed to directly minimize the difference of spectral information of each
band, which does not consider the inter-band relations in the training process.
In this letter, we propose a novel inter- and intra-band (IIB) loss to overcome
the drawback of original L2 loss. Our proposed IIB loss can effectively
preserve both inter- and intra-band relations and can be directly applied to
different pansharpening CNNs.
