  The computer vision community has seen a shift from convolutional-based to
pure transformer architectures for both image and video tasks. Training a
transformer from zero for these tasks usually requires a lot of data and
computational resources. Video Swin Transformer (VST) is a pure-transformer
model developed for video classification which achieves state-of-the-art
results in accuracy and efficiency on several datasets. In this paper, we aim
to understand if VST generalizes well enough to be used in an out-of-domain
setting. We study the performance of VST on two large-scale datasets, namely
FCVID and Something-Something using a transfer learning approach from
Kinetics-400, which requires around 4x less memory than training from scratch.
We then break down the results to understand where VST fails the most and in
which scenarios the transfer-learning approach is viable. Our experiments show
an 85\% top-1 accuracy on FCVID without retraining the whole model which is
equal to the state-of-the-art for the dataset and a 21\% accuracy on
Something-Something. The experiments also suggest that the performance of the
VST decreases on average when the video duration increases which seems to be a
consequence of a design choice of the model. From the results, we conclude that
VST generalizes well enough to classify out-of-domain videos without retraining
when the target classes are from the same type as the classes used to train the
model. We observed this effect when we performed transfer-learning from
Kinetics-400 to FCVID, where most datasets target mostly objects. On the other
hand, if the classes are not from the same type, then the accuracy after the
transfer-learning approach is expected to be poor. We observed this effect when
we performed transfer-learning from Kinetics-400, where the classes represent
mostly objects, to Something-Something, where the classes represent mostly
actions.
