  We introduce MarginMatch, a new SSL approach combining consistency
regularization and pseudo-labeling, with its main novelty arising from the use
of unlabeled data training dynamics to measure pseudo-label quality. Instead of
using only the model's confidence on an unlabeled example at an arbitrary
iteration to decide if the example should be masked or not, MarginMatch also
analyzes the behavior of the model on the pseudo-labeled examples as the
training progresses, to ensure low quality predictions are masked out.
MarginMatch brings substantial improvements on four vision benchmarks in low
data regimes and on two large-scale datasets, emphasizing the importance of
enforcing high-quality pseudo-labels. Notably, we obtain an improvement in
error rate over the state-of-the-art of 3.25% on CIFAR-100 with only 25 labels
per class and of 3.78% on STL-10 using as few as 4 labels per class. We make
our code available at https://github.com/tsosea2/MarginMatch.
