  Pseudo-labeling is a key component in semi-supervised learning (SSL). It
relies on iteratively using the model to generate artificial labels for the
unlabeled data to train against. A common property among its various methods is
that they only rely on the model's prediction to make labeling decisions
without considering any prior knowledge about the visual similarity among the
classes. In this paper, we demonstrate that this degrades the quality of
pseudo-labeling as it poorly represents visually similar classes in the pool of
pseudo-labeled data. We propose SemCo, a method which leverages label semantics
and co-training to address this problem. We train two classifiers with two
different views of the class labels: one classifier uses the one-hot view of
the labels and disregards any potential similarity among the classes, while the
other uses a distributed view of the labels and groups potentially similar
classes together. We then co-train the two classifiers to learn based on their
disagreements. We show that our method achieves state-of-the-art performance
across various SSL tasks including 5.6% accuracy improvement on Mini-ImageNet
dataset with 1000 labeled examples. We also show that our method requires
smaller batch size and fewer training iterations to reach its best performance.
We make our code available at https://github.com/islam-nassar/semco.
