  Transformers have achieved superior performances in many tasks in natural
language processing and computer vision, which also triggered great interest in
the time series community. Among multiple advantages of Transformers, the
ability to capture long-range dependencies and interactions is especially
attractive for time series modeling, leading to exciting progress in various
time series applications. In this paper, we systematically review Transformer
schemes for time series modeling by highlighting their strengths as well as
limitations. In particular, we examine the development of time series
Transformers in two perspectives. From the perspective of network structure, we
summarize the adaptations and modifications that have been made to Transformers
in order to accommodate the challenges in time series analysis. From the
perspective of applications, we categorize time series Transformers based on
common tasks including forecasting, anomaly detection, and classification.
Empirically, we perform robust analysis, model size analysis, and
seasonal-trend decomposition analysis to study how Transformers perform in time
series. Finally, we discuss and suggest future directions to provide useful
research guidance. To the best of our knowledge, this paper is the first work
to comprehensively and systematically summarize the recent advances of
Transformers for modeling time series data. We hope this survey will ignite
further research interests in time series Transformers.
