  It is commonly accepted that the Vision Transformer model requires
sophisticated regularization techniques to excel at ImageNet-1k scale data.
Surprisingly, we find this is not the case and standard data augmentation is
sufficient. This note presents a few minor modifications to the original Vision
Transformer (ViT) vanilla training setting that dramatically improve the
performance of plain ViT models. Notably, 90 epochs of training surpass 76%
top-1 accuracy in under seven hours on a TPUv3-8, similar to the classic
ResNet50 baseline, and 300 epochs of training reach 80% in less than one day.
