  Recurrent Networks are one of the most powerful and promising artificial
neural network algorithms to processing the sequential data such as natural
languages, sound, time series data. Unlike traditional feed-forward network,
Recurrent Network has a inherent feed back loop that allows to store the
temporal context information and pass the state of information to the entire
sequences of the events. This helps to achieve the state of art performance in
many important tasks such as language modeling, stock market prediction, image
captioning, speech recognition, machine translation and object tracking etc.,
However, training the fully connected RNN and managing the gradient flow are
the complicated process. Many studies are carried out to address the mentioned
limitation. This article is intent to provide the brief details about recurrent
neurons, its variances and trips & tricks to train the fully recurrent neural
network. This review work is carried out as a part of our IPO studio software
module 'Multiple Object Tracking'.
