  Time series forecasting with limited data is a challenging yet critical task.
While transformers have achieved outstanding performances in time series
forecasting, they often require many training samples due to the large number
of trainable parameters. In this paper, we propose a training technique for
transformers that prepares the training windows through random sampling. As
input time steps need not be consecutive, the number of distinct samples
increases from linearly to combinatorially many. By breaking the temporal
order, this technique also helps transformers to capture dependencies among
time steps in finer granularity. We achieve competitive results compared to the
state-of-the-art on real-world datasets.
