  Only parts of unlabeled data are selected to train models for most
semi-supervised learning methods, whose confidence scores are usually higher
than the pre-defined threshold (i.e., the confidence margin). We argue that the
recognition performance should be further improved by making full use of all
unlabeled data. In this paper, we learn an Adaptive Confidence Margin (Ada-CM)
to fully leverage all unlabeled data for semi-supervised deep facial expression
recognition. All unlabeled samples are partitioned into two subsets by
comparing their confidence scores with the adaptively learned confidence margin
at each training epoch: (1) subset I including samples whose confidence scores
are no lower than the margin; (2) subset II including samples whose confidence
scores are lower than the margin. For samples in subset I, we constrain their
predictions to match pseudo labels. Meanwhile, samples in subset II participate
in the feature-level contrastive objective to learn effective facial expression
features. We extensively evaluate Ada-CM on four challenging datasets, showing
that our method achieves state-of-the-art performance, especially surpassing
fully-supervised baselines in a semi-supervised manner. Ablation study further
proves the effectiveness of our method. The source code is available at
https://github.com/hangyu94/Ada-CM.
