  Deep learning models for multimodal expression recognition have reached
remarkable performance in controlled laboratory environments because of their
ability to learn complementary and redundant semantic information. However,
these models struggle in the wild, mainly because of the unavailability and
quality of modalities used for training. In practice, only a subset of the
training-time modalities may be available at test time. Learning with
privileged information enables models to exploit data from additional
modalities that are only available during training. State-of-the-art knowledge
distillation (KD) methods have been proposed to distill information from
multiple teacher models (each trained on a modality) to a common student model.
These privileged KD methods typically utilize point-to-point matching, yet have
no explicit mechanism to capture the structural information in the teacher
representation space formed by introducing the privileged modality. Experiments
were performed on two challenging problems - pain estimation on the Biovid
dataset (ordinal classification) and arousal-valance prediction on the Affwild2
dataset (regression). Results show that our proposed method can outperform
state-of-the-art privileged KD methods on these problems. The diversity among
modalities and fusion architectures indicates that PKDOT is modality- and
model-agnostic.
