  To minimize the effects of age variation in face recognition, previous work
either extracts identity-related discriminative features by minimizing the
correlation between identity- and age-related features, called age-invariant
face recognition (AIFR), or removes age variation by transforming the faces of
different age groups into the same age group, called face age synthesis (FAS);
however, the former lacks visual results for model interpretation while the
latter suffers from artifacts compromising downstream recognition. Therefore,
this paper proposes a unified, multi-task framework to jointly handle these two
tasks, termed MTLFace, which can learn age-invariant identity-related
representation while achieving pleasing face synthesis. Specifically, we first
decompose the mixed face feature into two uncorrelated components -- identity-
and age-related feature -- through an attention mechanism, and then decorrelate
these two components using multi-task training and continuous domain adaption.
In contrast to the conventional one-hot encoding that achieves group-level FAS,
we propose a novel identity conditional module to achieve identity-level FAS,
with a weight-sharing strategy to improve the age smoothness of synthesized
faces. In addition, we collect and release a large cross-age face dataset with
age and gender annotations to advance the development of the AIFR and FAS.
Extensive experiments on five benchmark cross-age datasets demonstrate the
superior performance of our proposed MTLFace over existing state-of-the-art
methods for AIFR and FAS. We further validate MTLFace on two popular general
face recognition datasets, showing competitive performance for face recognition
in the wild. The source code and dataset are available
at~\url{https://github.com/Hzzone/MTLFace}.
