  Heterogeneous face recognition (HFR) involves the intricate task of matching
face images across the visual domains of visible (VIS) and near-infrared (NIR).
While much of the existing literature on HFR identifies the domain gap as a
primary challenge and directs efforts towards bridging it at either the input
or feature level, our work deviates from this trend. We observe that large
neural networks, unlike their smaller counterparts, when pre-trained on large
scale homogeneous VIS data, demonstrate exceptional zero-shot performance in
HFR, suggesting that the domain gap might be less pronounced than previously
believed. By approaching the HFR problem as one of low-data fine-tuning, we
introduce a straightforward framework: comprehensive pre-training, succeeded by
a regularized fine-tuning strategy, that matches or surpasses the current
state-of-the-art on four publicly available benchmarks. Corresponding codes can
be found at https://github.com/michaeltrs/RethinkNIRVIS.
