  This work proposes a time series prediction method based on the kernel view
of linear reservoirs. In particular, the time series motifs of the reservoir
kernel are used as representational basis on which general readouts are
constructed. We provide a geometric interpretation of our approach shedding
light on how our approach is related to the core reservoir models and in what
way the two approaches differ. Empirical experiments then compare predictive
performances of our suggested model with those of recent state-of-art
transformer based models, as well as the established recurrent network model -
LSTM. The experiments are performed on both univariate and multivariate time
series and with a variety of prediction horizons. Rather surprisingly we show
that even when linear readout is employed, our method has the capacity to
outperform transformer models on univariate time series and attain competitive
results on multivariate benchmark datasets. We conclude that simple models with
easily controllable capacity but capturing enough memory and subsequence
structure can outperform potentially over-complicated deep learning models.
This does not mean that reservoir motif based models are preferable to other
more complex alternatives - rather, when introducing a new complex time series
model one should employ as a sanity check simple, but potentially powerful
alternatives/baselines such as reservoir models or the models introduced here.
