  Heterogeneous Face Recognition (HFR) refers to matching face images captured
in different domains, such as thermal to visible images (VIS), sketches to
visible images, near-infrared to visible, and so on. This is particularly
useful in matching visible spectrum images to images captured from other
modalities. Though highly useful, HFR is challenging because of the domain gap
between the source and target domain. Often, large-scale paired heterogeneous
face image datasets are absent, preventing training models specifically for the
heterogeneous task. In this work, we propose a surprisingly simple, yet, very
effective method for matching face images across different sensing modalities.
The core idea of the proposed approach is to add a novel neural network block
called Prepended Domain Transformer (PDT) in front of a pre-trained face
recognition (FR) model to address the domain gap. Retraining this new block
with few paired samples in a contrastive learning setup was enough to achieve
state-of-the-art performance in many HFR benchmarks. The PDT blocks can be
retrained for several source-target combinations using the proposed general
framework. The proposed approach is architecture agnostic, meaning they can be
added to any pre-trained FR models. Further, the approach is modular and the
new block can be trained with a minimal set of paired samples, making it much
easier for practical deployment. The source code and protocols will be made
available publicly.
