  Although significant advances have been made in face recognition (FR), FR in
unconstrained environments remains challenging due to the domain gap between
the semi-constrained training datasets and unconstrained testing scenarios. To
address this problem, we propose a controllable face synthesis model (CFSM)
that can mimic the distribution of target datasets in a style latent space.
CFSM learns a linear subspace with orthogonal bases in the style latent space
with precise control over the diversity and degree of synthesis. Furthermore,
the pre-trained synthesis model can be guided by the FR model, making the
resulting images more beneficial for FR model training. Besides, target dataset
distributions are characterized by the learned orthogonal bases, which can be
utilized to measure the distributional similarity among face datasets. Our
approach yields significant performance gains on unconstrained benchmarks, such
as IJB-B, IJB-C, TinyFace and IJB-S (+5.76% Rank1).
