  The recent advances in image transformers have shown impressive results and
have largely closed the gap between traditional CNN architectures. The standard
procedure is to train on large datasets like ImageNet-21k and then finetune on
ImageNet-1k. After finetuning, researches will often consider the transfer
learning performance on smaller datasets such as CIFAR-10/100 but have left out
Tiny ImageNet. This paper offers an update on vision transformers' performance
on Tiny ImageNet. I include Vision Transformer (ViT) , Data Efficient Image
Transformer (DeiT), Class Attention in Image Transformer (CaiT), and Swin
Transformers. In addition, Swin Transformers beats the current state-of-the-art
result with a validation accuracy of 91.35%. Code is available here:
https://github.com/ehuynh1106/TinyImageNet-Transformers
