  Pansharpening, a pivotal task in remote sensing, involves integrating
low-resolution multispectral images with high-resolution panchromatic images to
synthesize an image that is both high-resolution and retains multispectral
information. These pansharpened images enhance precision in land cover
classification, change detection, and environmental monitoring within remote
sensing data analysis. While deep learning techniques have shown significant
success in pansharpening, existing methods often face limitations in their
evaluation, focusing on restricted satellite data sources, single scene types,
and low-resolution images. This paper addresses this gap by introducing
PanBench, a high-resolution multi-scene dataset containing all mainstream
satellites and comprising 5,898 pairs of samples. Each pair includes a
four-channel (RGB + near-infrared) multispectral image of 256x256 pixels and a
mono-channel panchromatic image of 1,024x1,024 pixels. To achieve high-fidelity
synthesis, we propose a Cascaded Multiscale Fusion Network (CMFNet) for
Pansharpening. Extensive experiments validate the effectiveness of CMFNet. We
have released the dataset, source code, and pre-trained models in the
supplementary, fostering further research in remote sensing.
