  It is well known that the success of deep neural networks is greatly
attributed to large-scale labeled datasets. However, it can be extremely
time-consuming and laborious to collect sufficient high-quality labeled data in
most practical applications. Semi-supervised learning (SSL) provides an
effective solution to reduce the cost of labeling by simultaneously leveraging
both labeled and unlabeled data. In this work, we present Cross Labeling
Supervision (CLS), a framework that generalizes the typical pseudo-labeling
process. Based on FixMatch, where a pseudo label is generated from a
weakly-augmented sample to teach the prediction on a strong augmentation of the
same input sample, CLS allows the creation of both pseudo and complementary
labels to support both positive and negative learning. To mitigate the
confirmation bias of self-labeling and boost the tolerance to false labels, two
different initialized networks with the same structure are trained
simultaneously. Each network utilizes high-confidence labels from the other
network as additional supervision signals. During the label generation phase,
adaptive sample weights are assigned to artificial labels according to their
prediction confidence. The sample weight plays two roles: quantify the
generated labels' quality and reduce the disruption of inaccurate labels on
network training. Experimental results on the semi-supervised classification
task show that our framework outperforms existing approaches by large margins
on the CIFAR-10 and CIFAR-100 datasets.
