  Time series forecasting is prevalent in various real-world applications.
Despite the promising results of deep learning models in time series
forecasting, especially the Recurrent Neural Networks (RNNs), the explanations
of time series models, which are critical in high-stakes applications, have
received little attention. In this paper, we propose a Decomposition-based
Linear Explainable LSTM (DeLELSTM) to improve the interpretability of LSTM.
Conventionally, the interpretability of RNNs only concentrates on the variable
importance and time importance. We additionally distinguish between the
instantaneous influence of new coming data and the long-term effects of
historical data. Specifically, DeLELSTM consists of two components, i.e.,
standard LSTM and tensorized LSTM. The tensorized LSTM assigns each variable
with a unique hidden state making up a matrix $\mathbf{h}_t$, and the standard
LSTM models all the variables with a shared hidden state $\mathbf{H}_t$. By
decomposing the $\mathbf{H}_t$ into the linear combination of past information
$\mathbf{h}_{t-1}$ and the fresh information $\mathbf{h}_{t}-\mathbf{h}_{t-1}$,
we can get the instantaneous influence and the long-term effect of each
variable. In addition, the advantage of linear regression also makes the
explanation transparent and clear. We demonstrate the effectiveness and
interpretability of DeLELSTM on three empirical datasets. Extensive experiments
show that the proposed method achieves competitive performance against the
baseline methods and provides a reliable explanation relative to domain
knowledge.
