  Heterogeneous face recognition (HFR) refers to matching face images acquired
from different domains with wide applications in security scenarios. This paper
presents a deep neural network approach namely Multi-Margin based Decorrelation
Learning (MMDL) to extract decorrelation representations in a hyperspherical
space for cross-domain face images. The proposed framework can be divided into
two components: heterogeneous representation network and decorrelation
representation learning. First, we employ a large scale of accessible visual
face images to train heterogeneous representation network. The decorrelation
layer projects the output of the first component into decorrelation latent
subspace and obtains decorrelation representation. In addition, we design a
multi-margin loss (MML), which consists of quadruplet margin loss (QML) and
heterogeneous angular margin loss (HAML), to constrain the proposed framework.
Experimental results on two challenging heterogeneous face databases show that
our approach achieves superior performance on both verification and recognition
tasks, comparing with state-of-the-art methods.
