  The process of fusing a high spatial resolution (HR) panchromatic (PAN) image
and a low spatial resolution (LR) multispectral (MS) image to obtain an HRMS
image is known as pansharpening. With the development of convolutional neural
networks, the performance of pansharpening methods has been improved, however,
the blurry effects and the spectral distortion still exist in their fusion
results due to the insufficiency in details learning and the frequency mismatch
between MSand PAN. Therefore, the improvement of spatial details at the premise
of reducing spectral distortion is still a challenge. In this paper, we propose
a frequency-aware fusion network (FAFNet) together with a novel high-frequency
feature similarity loss to address above mentioned problems. FAFNet is mainly
composed of two kinds of blocks, where the frequency aware blocks aim to
extract features in the frequency domain with the help of discrete wavelet
transform (DWT) layers, and the frequency fusion blocks reconstruct and
transform the features from frequency domain to spatial domain with the
assistance of inverse DWT (IDWT) layers. Finally, the fusion results are
obtained through a convolutional block. In order to learn the correspondence,
we also propose a high-frequency feature similarity loss to constrain the HF
features derived from PAN and MS branches, so that HF features of PAN can
reasonably be used to supplement that of MS. Experimental results on three
datasets at both reduced- and full-resolution demonstrate the superiority of
the proposed method compared with several state-of-the-art pansharpening
models.
