  Surveillance cameras today often capture NIR (near infrared) images in
low-light environments. However, most face datasets accessible for training and
verification are only collected in the VIS (visible light) spectrum. It remains
a challenging problem to match NIR to VIS face images due to the different
light spectrum. Recently, breakthroughs have been made for VIS face recognition
by applying deep learning on a huge amount of labeled VIS face samples. The
same deep learning approach cannot be simply applied to NIR face recognition
for two main reasons: First, much limited NIR face images are available for
training compared to the VIS spectrum. Second, face galleries to be matched are
mostly available only in the VIS spectrum. In this paper, we propose an
approach to extend the deep learning breakthrough for VIS face recognition to
the NIR spectrum, without retraining the underlying deep models that see only
VIS faces. Our approach consists of two core components, cross-spectral
hallucination and low-rank embedding, to optimize respectively input and output
of a VIS deep model for cross-spectral face recognition. Cross-spectral
hallucination produces VIS faces from NIR images through a deep learning
approach. Low-rank embedding restores a low-rank structure for faces deep
features across both NIR and VIS spectrum. We observe that it is often equally
effective to perform hallucination to input NIR images or low-rank embedding to
output deep features for a VIS deep model for cross-spectral recognition. When
hallucination and low-rank embedding are deployed together, we observe
significant further improvement; we obtain state-of-the-art accuracy on the
CASIA NIR-VIS v2.0 benchmark, without the need at all to re-train the
recognition system.
