  Neural Machine Translation (NMT) can be improved by including document-level
contextual information. For this purpose, we propose a hierarchical attention
model to capture the context in a structured and dynamic manner. The model is
integrated in the original NMT architecture as another level of abstraction,
conditioning on the NMT model's own previous hidden states. Experiments show
that hierarchical attention significantly improves the BLEU score over a strong
NMT baseline with the state-of-the-art in context-aware methods, and that both
the encoder and decoder benefit from context in complementary ways.
