  Heterogeneous Face Recognition (HFR) is a challenging issue because of the
large domain discrepancy and a lack of heterogeneous data. This paper considers
HFR as a dual generation problem, and proposes a novel Dual Variational
Generation (DVG) framework. It generates large-scale new paired heterogeneous
images with the same identity from noise, for the sake of reducing the domain
gap of HFR. Specifically, we first introduce a dual variational autoencoder to
represent a joint distribution of paired heterogeneous images. Then, in order
to ensure the identity consistency of the generated paired heterogeneous
images, we impose a distribution alignment in the latent space and a pairwise
identity preserving in the image space. Moreover, the HFR network reduces the
domain discrepancy by constraining the pairwise feature distances between the
generated paired heterogeneous images. Extensive experiments on four HFR
databases show that our method can significantly improve state-of-the-art
results. The related code is available at https://github.com/BradyFU/DVG.
