  Heavy computation is a bottleneck limiting deep-learningbased feature
matching algorithms to be applied in many realtime applications. However,
existing lightweight networks optimized for Euclidean data cannot address
classical feature matching tasks, since sparse keypoint based descriptors are
expected to be matched. This paper tackles this problem and proposes two
concepts: 1) a novel parallel attention model entitled ParaFormer and 2) a
graph based U-Net architecture with attentional pooling. First, ParaFormer
fuses features and keypoint positions through the concept of amplitude and
phase, and integrates self- and cross-attention in a parallel manner which
achieves a win-win performance in terms of accuracy and efficiency. Second,
with U-Net architecture and proposed attentional pooling, the ParaFormer-U
variant significantly reduces computational complexity, and minimize
performance loss caused by downsampling. Sufficient experiments on various
applications, including homography estimation, pose estimation, and image
matching, demonstrate that ParaFormer achieves state-of-the-art performance
while maintaining high efficiency. The efficient ParaFormer-U variant achieves
comparable performance with less than 50% FLOPs of the existing attention-based
models.
