  The activations of Facial Action Units (AUs) mutually influence one another.
While the relationship between a pair of AUs can be complex and unique,
existing approaches fail to specifically and explicitly represent such cues for
each pair of AUs in each facial display. This paper proposes an AU relationship
modelling approach that deep learns a unique graph to explicitly describe the
relationship between each pair of AUs of the target facial display. Our
approach first encodes each AU's activation status and its association with
other AUs into a node feature. Then, it learns a pair of multi-dimensional edge
features to describe multiple task-specific relationship cues between each pair
of AUs. During both node and edge feature learning, our approach also considers
the influence of the unique facial display on AUs' relationship by taking the
full face representation as an input. Experimental results on BP4D and DISFA
datasets show that both node and edge feature learning modules provide large
performance improvements for CNN and transformer-based backbones, with our best
systems achieving the state-of-the-art AU recognition results. Our approach not
only has a strong capability in modelling relationship cues for AU recognition
but also can be easily incorporated into various backbones. Our PyTorch code is
made available.
