  Pansharpening in remote sensing image aims at acquiring a high-resolution
multispectral (HRMS) image directly by fusing a low-resolution multispectral
(LRMS) image with a panchromatic (PAN) image. The main concern is how to
effectively combine the rich spectral information of LRMS image with the
abundant spatial information of PAN image. Recently, many methods based on deep
learning have been proposed for the pansharpening task. However, these methods
usually has two main drawbacks: 1) requiring HRMS for supervised learning; and
2) simply ignoring the latent relation between the MS and PAN image and fusing
them directly. To solve these problems, we propose a novel unsupervised network
based on learnable degradation processes, dubbed as LDP-Net. A reblurring block
and a graying block are designed to learn the corresponding degradation
processes, respectively. In addition, a novel hybrid loss function is proposed
to constrain both spatial and spectral consistency between the pansharpened
image and the PAN and LRMS images at different resolutions. Experiments on
Worldview2 and Worldview3 images demonstrate that our proposed LDP-Net can fuse
PAN and LRMS images effectively without the help of HRMS samples, achieving
promising performance in terms of both qualitative visual effects and
quantitative metrics.
