  Recurrent neural network architectures combining with attention mechanism, or
neural attention model, have shown promising performance recently for the tasks
including speech recognition, image caption generation, visual question
answering and machine translation. In this paper, neural attention model is
applied on two sequence classification tasks, dialogue act detection and key
term extraction. In the sequence labeling tasks, the model input is a sequence,
and the output is the label of the input sequence. The major difficulty of
sequence labeling is that when the input sequence is long, it can include many
noisy or irrelevant part. If the information in the whole sequence is treated
equally, the noisy or irrelevant part may degrade the classification
performance. The attention mechanism is helpful for sequence classification
task because it is capable of highlighting important part among the entire
sequence for the classification task. The experimental results show that with
the attention mechanism, discernible improvements were achieved in the sequence
labeling task considered here. The roles of the attention mechanism in the
tasks are further analyzed and visualized in this paper.
