  Facial expression recognition (FER), aiming to classify the expression
present in the facial image or video, has attracted a lot of research interests
in the field of artificial intelligence and multimedia. In terms of video based
FER task, it is sensible to capture the dynamic expression variation among the
frames to recognize facial expression. However, existing methods directly
utilize CNN-RNN or 3D CNN to extract the spatial-temporal features from
different facial units, instead of concentrating on a certain region during
expression variation capturing, which leads to limited performance in FER. In
our paper, we introduce a Graph Convolutional Network (GCN) layer into a common
CNN-RNN based model for video-based FER. First, the GCN layer is utilized to
learn more significant facial expression features which concentrate on certain
regions after sharing information between extracted CNN features of nodes.
Then, a LSTM layer is applied to learn long-term dependencies among the GCN
learned features to model the variation. In addition, a weight assignment
mechanism is also designed to weight the output of different nodes for final
classification by characterizing the expression intensities in each frame. To
the best of our knowledge, it is the first time to use GCN in FER task. We
evaluate our method on three widely-used datasets, CK+, Oulu-CASIA and MMI, and
also one challenging wild dataset AFEW8.0, and the experimental results
demonstrate that our method has superior performance to existing methods.
