  In this paper we introduce the Temporo-Spatial Vision Transformer (TSViT), a
fully-attentional model for general Satellite Image Time Series (SITS)
processing based on the Vision Transformer (ViT). TSViT splits a SITS record
into non-overlapping patches in space and time which are tokenized and
subsequently processed by a factorized temporo-spatial encoder. We argue, that
in contrast to natural images, a temporal-then-spatial factorization is more
intuitive for SITS processing and present experimental evidence for this claim.
Additionally, we enhance the model's discriminative power by introducing two
novel mechanisms for acquisition-time-specific temporal positional encodings
and multiple learnable class tokens. The effect of all novel design choices is
evaluated through an extensive ablation study. Our proposed architecture
achieves state-of-the-art performance, surpassing previous approaches by a
significant margin in three publicly available SITS semantic segmentation and
classification datasets. All model, training and evaluation codes are made
publicly available to facilitate further research.
