  Previous methods for dynamic facial expression in the wild are mainly based
on Convolutional Neural Networks (CNNs), whose local operations ignore the
long-range dependencies in videos. To solve this problem, we propose the
spatio-temporal Transformer (STT) to capture discriminative features within
each frame and model contextual relationships among frames. Spatio-temporal
dependencies are captured and integrated by our unified Transformer.
Specifically, given an image sequence consisting of multiple frames as input,
we utilize the CNN backbone to translate each frame into a visual feature
sequence. Subsequently, the spatial attention and the temporal attention within
each block are jointly applied for learning spatio-temporal representations at
the sequence level. In addition, we propose the compact softmax cross entropy
loss to further encourage the learned features have the minimum intra-class
distance and the maximum inter-class distance. Experiments on two in-the-wild
dynamic facial expression datasets (i.e., DFEW and AFEW) indicate that our
method provides an effective way to make use of the spatial and temporal
dependencies for dynamic facial expression recognition. The source code and the
training logs will be made publicly available.
