  Recurrent and convolutional neural networks are the most common architectures
used for time series forecasting in deep learning literature. These networks
use parameter sharing by repeating a set of fixed architectures with fixed
parameters over time or space. The result is that the overall architecture is
time-invariant (shift-invariant in the spatial domain) or stationary. We argue
that time-invariance can reduce the capacity to perform multi-step-ahead
forecasting, where modelling the dynamics at a range of scales and resolutions
is required. We propose ForecastNet which uses a deep feed-forward architecture
to provide a time-variant model. An additional novelty of ForecastNet is
interleaved outputs, which we show assist in mitigating vanishing gradients.
ForecastNet is demonstrated to outperform statistical and deep learning
benchmark models on several datasets.
