  Despite the success of deep neural networks on facial action unit (AU)
detection, better performance depends on a large number of training images with
accurate AU annotations. However, labeling AU is time-consuming, expensive, and
error-prone. Considering AU detection and facial expression recognition (FER)
are two highly correlated tasks, and facial expression (FE) is relatively easy
to annotate, we consider learning AU detection and FER in a multi-task manner.
However, the performance of the AU detection task cannot be always enhanced due
to the negative transfer in the multi-task scenario. To alleviate this issue,
we propose a Meta Auxiliary Learning method (MAL) that automatically selects
highly related FE samples by learning adaptative weights for the training FE
samples in a meta learning manner. The learned sample weights alleviate the
negative transfer from two aspects: 1) balance the loss of each task
automatically, and 2) suppress the weights of FE samples that have large
uncertainties. Experimental results on several popular AU datasets demonstrate
MAL consistently improves the AU detection performance compared with the
state-of-the-art multi-task and auxiliary learning methods. MAL automatically
estimates adaptive weights for the auxiliary FE samples according to their
semantic relevance with the primary AU detection task.
