  In this paper, we propose an interpretable LSTM recurrent neural network,
i.e., multi-variable LSTM for time series with exogenous variables. Currently,
widely used attention mechanism in recurrent neural networks mostly focuses on
the temporal aspect of data and falls short of characterizing variable
importance. To this end, our multi-variable LSTM equipped with tensorized
hidden states is developed to learn variable specific representations, which
give rise to both temporal and variable level attention. Preliminary
experiments demonstrate comparable prediction performance of multi-variable
LSTM w.r.t. encoder-decoder based baselines. More interestingly, variable
importance in real datasets characterized by the variable attention is highly
in line with that determined by statistical Granger causality test, which
exhibits the prospect of multi-variable LSTM as a simple and uniform end-to-end
framework for both forecasting and knowledge discovery.
