  Recurrent Neural Networks (RNNs) have become increasingly popular for the
task of language understanding. In this task, a semantic tagger is deployed to
associate a semantic label to each word in an input sequence. The success of
RNN may be attributed to its ability to memorize long-term dependence that
relates the current-time semantic label prediction to the observations many
time instances away. However, the memory capacity of simple RNNs is limited
because of the gradient vanishing and exploding problem. We propose to use an
external memory to improve memorization capability of RNNs. We conducted
experiments on the ATIS dataset, and observed that the proposed model was able
to achieve the state-of-the-art results. We compare our proposed model with
alternative models and report analysis results that may provide insights for
future research.
