  Recurrent Neural Networks (RNNs) have been widely applied to deal with
temporal problems, such as flood forecasting and financial data processing. On
the one hand, traditional RNNs models amplify the gradient issue due to the
strict time serial dependency, making it difficult to realize a long-term
memory function. On the other hand, RNNs cells are highly complex, which will
significantly increase computational complexity and cause waste of
computational resources during model training. In this paper, an improved Time
Feedforward Connections Recurrent Neural Networks (TFC-RNNs) model was first
proposed to address the gradient issue. A parallel branch was introduced for
the hidden state at time t-2 to be directly transferred to time t without the
nonlinear transformation at time t-1. This is effective in improving the
long-term dependence of RNNs. Then, a novel cell structure named Single Gate
Recurrent Unit (SGRU) was presented. This cell structure can reduce the number
of parameters for RNNs cell, consequently reducing the computational
complexity. Next, applying SGRU to TFC-RNNs as a new TFC-SGRU model solves the
above two difficulties. Finally, the performance of our proposed TFC-SGRU was
verified through several experiments in terms of long-term memory and
anti-interference capabilities. Experimental results demonstrated that our
proposed TFC-SGRU model can capture helpful information with time step 1500 and
effectively filter out the noise. The TFC-SGRU model accuracy is better than
the LSTM and GRU models regarding language processing ability.
