  This paper explores multi-task learning (MTL) for face recognition. We answer
the questions of how and why MTL can improve the face recognition performance.
First, we propose a multi-task Convolutional Neural Network (CNN) for face
recognition where identity classification is the main task and pose,
illumination, and expression estimations are the side tasks. Second, we develop
a dynamic-weighting scheme to automatically assign the loss weight to each side
task, which is a crucial problem in MTL. Third, we propose a pose-directed
multi-task CNN by grouping different poses to learn pose-specific identity
features, simultaneously across all poses. Last but not least, we propose an
energy-based weight analysis method to explore how CNN-based MTL works. We
observe that the side tasks serve as regularizations to disentangle the
variations from the learnt identity features. Extensive experiments on the
entire Multi-PIE dataset demonstrate the effectiveness of the proposed
approach. To the best of our knowledge, this is the first work using all data
in Multi-PIE for face recognition. Our approach is also applicable to
in-the-wild datasets for pose-invariant face recognition and achieves
comparable or better performance than state of the art on LFW, CFP, and IJB-A
datasets.
