  Transformers have recently gained increasing attention in computer vision.
However, existing studies mostly use Transformers for feature representation
learning, e.g. for image classification and dense predictions, and the
generalizability of Transformers is unknown. In this work, we further
investigate the possibility of applying Transformers for image matching and
metric learning given pairs of images. We find that the Vision Transformer
(ViT) and the vanilla Transformer with decoders are not adequate for image
matching due to their lack of image-to-image attention. Thus, we further design
two naive solutions, i.e. query-gallery concatenation in ViT, and query-gallery
cross-attention in the vanilla Transformer. The latter improves the
performance, but it is still limited. This implies that the attention mechanism
in Transformers is primarily designed for global feature aggregation, which is
not naturally suitable for image matching. Accordingly, we propose a new
simplified decoder, which drops the full attention implementation with the
softmax weighting, keeping only the query-key similarity computation.
Additionally, global max pooling and a multilayer perceptron (MLP) head are
applied to decode the matching result. This way, the simplified decoder is
computationally more efficient, while at the same time more effective for image
matching. The proposed method, called TransMatcher, achieves state-of-the-art
performance in generalizable person re-identification, with up to 6.1% and 5.7%
performance gains in Rank-1 and mAP, respectively, on several popular datasets.
Code is available at https://github.com/ShengcaiLiao/QAConv.
