  Deep learning technology can be used as an assistive technology to help
doctors quickly and accurately identify COVID-19 infections. Recently, Vision
Transformer (ViT) has shown great potential towards image classification due to
its global receptive field. However, due to the lack of inductive biases
inherent to CNNs, the ViT-based structure leads to limited feature richness and
difficulty in model training. In this paper, we propose a new structure called
Transformer for COVID-19 (COVT) to improve the performance of ViT-based
architectures on small COVID-19 datasets. It uses CNN as a feature extractor to
effectively extract local structural information, and introduces average
pooling to ViT's Multilayer Perception(MLP) module for global information.
Experiments show the effectiveness of our method on the two COVID-19 datasets
and the ImageNet dataset.
