  Heterogeneous Face Recognition (HFR) refers to matching cross-domain faces
and plays a crucial role in public security. Nevertheless, HFR is confronted
with challenges from large domain discrepancy and insufficient heterogeneous
data. In this paper, we formulate HFR as a dual generation problem, and tackle
it via a novel Dual Variational Generation (DVG-Face) framework. Specifically,
a dual variational generator is elaborately designed to learn the joint
distribution of paired heterogeneous images. However, the small-scale paired
heterogeneous training data may limit the identity diversity of sampling. In
order to break through the limitation, we propose to integrate abundant
identity information of large-scale visible data into the joint distribution.
Furthermore, a pairwise identity preserving loss is imposed on the generated
paired heterogeneous images to ensure their identity consistency. As a
consequence, massive new diverse paired heterogeneous images with the same
identity can be generated from noises. The identity consistency and identity
diversity properties allow us to employ these generated images to train the HFR
network via a contrastive learning mechanism, yielding both domain-invariant
and discriminative embedding features. Concretely, the generated paired
heterogeneous images are regarded as positive pairs, and the images obtained
from different samplings are considered as negative pairs. Our method achieves
superior performances over state-of-the-art methods on seven challenging
databases belonging to five HFR tasks, including NIR-VIS, Sketch-Photo,
Profile-Frontal Photo, Thermal-VIS, and ID-Camera. The related code will be
released at https://github.com/BradyFU.
