  Semi-supervised learning (SSL) is an effective means to leverage unlabeled
data to improve a model's performance. Typical SSL methods like FixMatch assume
that labeled and unlabeled data share the same label space. However, in
practice, unlabeled data can contain categories unseen in the labeled set,
i.e., outliers, which can significantly harm the performance of SSL algorithms.
To address this problem, we propose a novel Open-set Semi-Supervised Learning
(OSSL) approach called OpenMatch. Learning representations of inliers while
rejecting outliers is essential for the success of OSSL. To this end, OpenMatch
unifies FixMatch with novelty detection based on one-vs-all (OVA) classifiers.
The OVA-classifier outputs the confidence score of a sample being an inlier,
providing a threshold to detect outliers. Another key contribution is an
open-set soft-consistency regularization loss, which enhances the smoothness of
the OVA-classifier with respect to input transformations and greatly improves
outlier detection. OpenMatch achieves state-of-the-art performance on three
datasets, and even outperforms a fully supervised model in detecting outliers
unseen in unlabeled data on CIFAR10.
