  Recurrent Neural Networks (RNNs) and their variants, such as Long-Short Term
Memory (LSTM) networks, and Gated Recurrent Unit (GRU) networks, have achieved
promising performance in sequential data modeling. The hidden layers in RNNs
can be regarded as the memory units, which are helpful in storing information
in sequential contexts. However, when dealing with high dimensional input data,
such as video and text, the input-to-hidden linear transformation in RNNs
brings high memory usage and huge computational cost. This makes the training
of RNNs unscalable and difficult. To address this challenge, we propose a novel
compact LSTM model, named as TR-LSTM, by utilizing the low-rank tensor ring
decomposition (TRD) to reformulate the input-to-hidden transformation. Compared
with other tensor decomposition methods, TR-LSTM is more stable. In addition,
TR-LSTM can complete an end-to-end training and also provide a fundamental
building block for RNNs in handling large input data. Experiments on real-world
action recognition datasets have demonstrated the promising performance of the
proposed TR-LSTM compared with the tensor train LSTM and other state-of-the-art
competitors.
