  Recently, deep learning techniques have been extensively studied for
pansharpening, which aims to generate a high resolution multispectral (HRMS)
image by fusing a low resolution multispectral (LRMS) image with a high
resolution panchromatic (PAN) image. However, existing deep learning-based
pansharpening methods directly learn the mapping from LRMS and PAN to HRMS.
These network architectures always lack sufficient interpretability, which
limits further performance improvements. To alleviate this issue, we propose a
novel deep network for pansharpening by combining the model-based methodology
with the deep learning method. Firstly, we build an observation model for
pansharpening using the convolutional sparse coding (CSC) technique and design
a proximal gradient algorithm to solve this model. Secondly, we unfold the
iterative algorithm into a deep network, dubbed as Proximal PanNet, by learning
the proximal operators using convolutional neural networks. Finally, all the
learnable modules can be automatically learned in an end-to-end manner.
Experimental results on some benchmark datasets show that our network performs
better than other advanced methods both quantitatively and qualitatively.
