  Spatiotemporal forecasting techniques are significant for various domains
such as transportation, energy, and weather. Accurate prediction of
spatiotemporal series remains challenging due to the complex spatiotemporal
heterogeneity. In particular, current end-to-end models are limited by input
length and thus often fall into spatiotemporal mirage, i.e., similar input time
series followed by dissimilar future values and vice versa. To address these
problems, we propose a novel self-supervised pre-training framework
Spatial-Temporal-Decoupled Masked Pre-training (STD-MAE) that employs two
decoupled masked autoencoders to reconstruct spatiotemporal series along the
spatial and temporal dimensions. Rich-context representations learned through
such reconstruction could be seamlessly integrated by downstream predictors
with arbitrary architectures to augment their performances. A series of
quantitative and qualitative evaluations on six widely used benchmarks (PEMS03,
PEMS04, PEMS07, PEMS08, METR-LA, and PEMS-BAY) are conducted to validate the
state-of-the-art performance of STD-MAE. Codes are available at
https://github.com/Jimmy-7664/STD-MAE.
