  Compared with facial emotion recognition on categorical model, the
dimensional emotion recognition can describe numerous emotions of the real
world more accurately. Most prior works of dimensional emotion estimation only
considered laboratory data and used video, speech or other multi-modal
features. The effect of these methods applied on static images in the real
world is unknown. In this paper, a two-level attention with two-stage
multi-task learning (2Att-2Mt) framework is proposed for facial emotion
estimation on only static images. Firstly, the features of corresponding
region(position-level features) are extracted and enhanced automatically by
first-level attention mechanism. In the following, we utilize Bi-directional
Recurrent Neural Network(Bi-RNN) with self-attention(second-level attention) to
make full use of the relationship features of different layers(layer-level
features) adaptively. Owing to the inherent complexity of dimensional emotion
recognition, we propose a two-stage multi-task learning structure to exploited
categorical representations to ameliorate the dimensional representations and
estimate valence and arousal simultaneously in view of the correlation of the
two targets. The quantitative results conducted on AffectNet dataset show
significant advancement on Concordance Correlation Coefficient(CCC) and Root
Mean Square Error(RMSE), illustrating the superiority of the proposed
framework. Besides, extensive comparative experiments have also fully
demonstrated the effectiveness of different components.
