  Holistic methods using CNNs and margin-based losses have dominated research
on face recognition. In this work, we depart from this setting in two ways: (a)
we employ the Vision Transformer as an architecture for training a very strong
baseline for face recognition, simply called fViT, which already surpasses most
state-of-the-art face recognition methods. (b) Secondly, we capitalize on the
Transformer's inherent property to process information (visual tokens)
extracted from irregular grids to devise a pipeline for face recognition which
is reminiscent of part-based face recognition methods. Our pipeline, called
part fViT, simply comprises a lightweight network to predict the coordinates of
facial landmarks followed by the Vision Transformer operating on patches
extracted from the predicted landmarks, and it is trained end-to-end with no
landmark supervision. By learning to extract discriminative patches, our
part-based Transformer further boosts the accuracy of our Vision Transformer
baseline achieving state-of-the-art accuracy on several face recognition
benchmarks.
