  Image matching aims at identifying corresponding points between a pair of
images. Currently, detector-free methods have shown impressive performance in
challenging scenarios, thanks to their capability of generating dense matches
and global receptive field. However, performing feature interaction and
proposing matches across the entire image is unnecessary, because not all image
regions contribute to the matching process. Interacting and matching in
unmatchable areas can introduce errors, reducing matching accuracy and
efficiency. Meanwhile, the scale discrepancy issue still troubles existing
methods. To address above issues, we propose PRogressive dependency
maxImization for Scale-invariant image Matching (PRISM), which jointly prunes
irrelevant patch features and tackles the scale discrepancy. To do this, we
firstly present a Multi-scale Pruning Module (MPM) to adaptively prune
irrelevant features by maximizing the dependency between the two feature sets.
Moreover, we design the Scale-Aware Dynamic Pruning Attention (SADPA) to
aggregate information from different scales via a hierarchical design. Our
method's superior matching performance and generalization capability are
confirmed by leading accuracy across various evaluation benchmarks and
downstream tasks. The code is publicly available at
https://github.com/Master-cai/PRISM.
