  Prior work has shown that the order in which different components of the face
are learned using a sequential learner can play an important role in the
performance of facial expression recognition systems. We propose FaceTopoNet,
an end-to-end deep model for facial expression recognition, which is capable of
learning an effective tree topology of the face. Our model then traverses the
learned tree to generate a sequence, which is then used to form an embedding to
feed a sequential learner. The devised model adopts one stream for learning
structure and one stream for learning texture. The structure stream focuses on
the positions of the facial landmarks, while the main focus of the texture
stream is on the patches around the landmarks to learn textural information. We
then fuse the outputs of the two streams by utilizing an effective
attention-based fusion strategy. We perform extensive experiments on four
large-scale in-the-wild facial expression datasets - namely AffectNet, FER2013,
ExpW, and RAF-DB - and one lab-controlled dataset (CK+) to evaluate our
approach. FaceTopoNet achieves state-of-the-art performance on three of the
five datasets and obtains competitive results on the other two datasets. We
also perform rigorous ablation and sensitivity experiments to evaluate the
impact of different components and parameters in our model. Lastly, we perform
robustness experiments and demonstrate that FaceTopoNet is more robust against
occlusions in comparison to other leading methods in the area.
