  Facial expression recognition (FER) is a challenging problem because the
expression component is always entangled with other irrelevant factors, such as
identity and head pose. In this work, we propose an identity and pose
disentangled facial expression recognition (IPD-FER) model to learn more
discriminative feature representation. We regard the holistic facial
representation as the combination of identity, pose and expression. These three
components are encoded with different encoders. For identity encoder, a well
pre-trained face recognition model is utilized and fixed during training, which
alleviates the restriction on specific expression training data in previous
works and makes the disentanglement practicable on in-the-wild datasets. At the
same time, the pose and expression encoder are optimized with corresponding
labels. Combining identity and pose feature, a neutral face of input individual
should be generated by the decoder. When expression feature is added, the input
image should be reconstructed. By comparing the difference between synthesized
neutral and expressional images of the same individual, the expression
component is further disentangled from identity and pose. Experimental results
verify the effectiveness of our method on both lab-controlled and in-the-wild
databases and we achieve state-of-the-art recognition performance.
