  Automated emotion recognition in the wild from facial images remains a
challenging problem. Although recent advances in Deep Learning have supposed a
significant breakthrough in this topic, strong changes in pose, orientation and
point of view severely harm current approaches. In addition, the acquisition of
labeled datasets is costly, and current state-of-the-art deep learning
algorithms cannot model all the aforementioned difficulties. In this paper, we
propose to apply a multi-task learning loss function to share a common feature
representation with other related tasks. Particularly we show that emotion
recognition benefits from jointly learning a model with a detector of facial
Action Units (collective muscle movements). The proposed loss function
addresses the problem of learning multiple tasks with heterogeneously labeled
data, improving previous multi-task approaches. We validate the proposal using
two datasets acquired in non controlled environments, and an application to
predict compound facial emotion expressions.
