  Facial Expression Recognition (FER) in the wild is an extremely challenging
task. Recently, some Vision Transformers (ViT) have been explored for FER, but
most of them perform inferiorly compared to Convolutional Neural Networks
(CNN). This is mainly because the new proposed modules are difficult to
converge well from scratch due to lacking inductive bias and easy to focus on
the occlusion and noisy areas. TransFER, a representative transformer-based
method for FER, alleviates this with multi-branch attention dropping but brings
excessive computations. On the contrary, we present two attentive pooling (AP)
modules to pool noisy features directly. The AP modules include Attentive Patch
Pooling (APP) and Attentive Token Pooling (ATP). They aim to guide the model to
emphasize the most discriminative features while reducing the impacts of less
relevant features. The proposed APP is employed to select the most informative
patches on CNN features, and ATP discards unimportant tokens in ViT. Being
simple to implement and without learnable parameters, the APP and ATP
intuitively reduce the computational cost while boosting the performance by
ONLY pursuing the most discriminative features. Qualitative results demonstrate
the motivations and effectiveness of our attentive poolings. Besides,
quantitative results on six in-the-wild datasets outperform other
state-of-the-art methods.
