  The Transformer architecture has witnessed a rapid development in recent
years, outperforming the CNN architectures in many computer vision tasks, as
exemplified by the Vision Transformers (ViT) for image classification. However,
existing visual transformer models aim to extract semantic information for
high-level tasks, such as classification and detection.These methods ignore the
importance of the spatial resolution of the input image, thus sacrificing the
local correlation information of neighboring pixels. In this paper, we propose
a Patch Pyramid Transformer(PPT) to effectively address the above
issues.Specifically, we first design a Patch Transformer to transform the image
into a sequence of patches, where transformer encoding is performed for each
patch to extract local representations. In addition, we construct a Pyramid
Transformer to effectively extract the non-local information from the entire
image. After obtaining a set of multi-scale, multi-dimensional, and multi-angle
features of the original image, we design the image reconstruction network to
ensure that the features can be reconstructed into the original input. To
validate the effectiveness, we apply the proposed Patch Pyramid Transformer to
image fusion tasks. The experimental results demonstrate its superior
performance, compared to the state-of-the-art fusion approaches, achieving the
best results on several evaluation indicators. Thanks to the underlying
representational capacity of the PPT network, it can directly be applied to
different image fusion tasks without redesigning or retraining the network.
