  We study the training of Vision Transformers for semi-supervised image
classification. Transformers have recently demonstrated impressive performance
on a multitude of supervised learning tasks. Surprisingly, we show Vision
Transformers perform significantly worse than Convolutional Neural Networks
when only a small set of labeled data is available. Inspired by this
observation, we introduce a joint semi-supervised learning framework,
Semiformer, which contains a transformer stream, a convolutional stream and a
carefully designed fusion module for knowledge sharing between these streams.
The convolutional stream is trained on limited labeled data and further used to
generate pseudo labels to supervise the training of the transformer stream on
unlabeled data. Extensive experiments on ImageNet demonstrate that Semiformer
achieves 75.5% top-1 accuracy, outperforming the state-of-the-art by a clear
margin. In addition, we show, among other things, Semiformer is a general
framework that is compatible with most modern transformer and convolutional
neural architectures. Code is available at
https://github.com/wengzejia1/Semiformer.
