  We describe an application of an encoder-decoder recurrent neural network
with LSTM units and attention to generating headlines from the text of news
articles. We find that the model is quite effective at concisely paraphrasing
news articles. Furthermore, we study how the neural network decides which input
words to pay attention to, and specifically we identify the function of the
different neurons in a simplified attention mechanism. Interestingly, our
simplified attention mechanism performs better that the more complex attention
mechanism on a held out set of articles.
