  The vision community is witnessing a modeling shift from CNNs to
Transformers, where pure Transformer architectures have attained top accuracy
on the major video recognition benchmarks. These video models are all built on
Transformer layers that globally connect patches across the spatial and
temporal dimensions. In this paper, we instead advocate an inductive bias of
locality in video Transformers, which leads to a better speed-accuracy
trade-off compared to previous approaches which compute self-attention globally
even with spatial-temporal factorization. The locality of the proposed video
architecture is realized by adapting the Swin Transformer designed for the
image domain, while continuing to leverage the power of pre-trained image
models. Our approach achieves state-of-the-art accuracy on a broad range of
video recognition benchmarks, including on action recognition (84.9 top-1
accuracy on Kinetics-400 and 86.1 top-1 accuracy on Kinetics-600 with ~20x less
pre-training data and ~3x smaller model size) and temporal modeling (69.6 top-1
accuracy on Something-Something v2). The code and models will be made publicly
available at https://github.com/SwinTransformer/Video-Swin-Transformer.
