  In the machine learning fields, Recurrent Neural Network (RNN) has become a
popular architecture for sequential data modeling. However, behind the
impressive performance, RNNs require a large number of parameters for both
training and inference. In this paper, we are trying to reduce the number of
parameters and maintain the expressive power from RNN simultaneously. We
utilize several tensor decompositions method including CANDECOMP/PARAFAC (CP),
Tucker decomposition and Tensor Train (TT) to re-parameterize the Gated
Recurrent Unit (GRU) RNN. We evaluate all tensor-based RNNs performance on
sequence modeling tasks with a various number of parameters. Based on our
experiment results, TT-GRU achieved the best results in a various number of
parameters compared to other decomposition methods.
