  This paper proposes to generalize the variational recurrent neural network
(RNN) with variational inference (VI)-based dropout regularization employed for
the long short-term memory (LSTM) cells to more advanced RNN architectures like
gated recurrent unit (GRU) and bi-directional LSTM/GRU. The new variational
RNNs are employed for slot filling, which is an intriguing but challenging task
in spoken language understanding. The experiments on the ATIS dataset suggest
that the variational RNNs with the VI-based dropout regularization can
significantly improve the naive dropout regularization RNNs-based baseline
systems in terms of F-measure. Particularly, the variational RNN with
bi-directional LSTM/GRU obtains the best F-measure score.
