  Recurrent Neural Networks (RNNs) produce state-of-art performance on many
machine learning tasks but their demand on resources in terms of memory and
computational power are often high. Therefore, there is a great interest in
optimizing the computations performed with these models especially when
considering development of specialized low-power hardware for deep networks.
One way of reducing the computational needs is to limit the numerical precision
of the network weights and biases, and this will be addressed for the case of
RNNs. We present results from the use of different stochastic and deterministic
reduced precision training methods applied to two major RNN types, which are
then tested on three datasets. The results show that the stochastic and
deterministic ternarization, pow2- ternarization, and exponential quantization
methods gave rise to low-precision RNNs that produce similar and even higher
accuracy on certain datasets, therefore providing a path towards training more
efficient implementations of RNNs in specialized hardware.
