  Automatic emotion recognition for real-life appli-cations is a challenging
task. Human emotion expressions aresubtle, and can be conveyed by a combination
of several emo-tions. In most existing emotion recognition studies, each
audioutterance/video clip is labelled/classified in its entirety.
However,utterance/clip-level labelling and classification can be too coarseto
capture the subtle intra-utterance/clip temporal dynamics. Forexample, an
utterance/video clip usually contains only a fewemotion-salient regions and
many emotionless regions. In thisstudy, we propose to use attention mechanism
in deep recurrentneural networks to detection the Regions-of-Interest (ROI)
thatare more emotionally salient in human emotional speech/video,and further
estimate the temporal emotion dynamics by aggre-gating those emotionally
salient regions-of-interest. We comparethe ROI from audio and video and analyse
them. We comparethe performance of the proposed attention networks with
thestate-of-the-art LSTM models on multi-class classification task
ofrecognizing six basic human emotions, and the proposed attentionmodels
exhibit significantly better performance. Furthermore, theattention weight
distribution can be used to interpret how anutterance can be expressed as a
mixture of possible emotions.
