  Attention plays a key role in the improvement of sequence-to-sequence-based
document summarization models. To obtain a powerful attention helping with
reproducing the most salient information and avoiding repetitions, we augment
the vanilla attention model from both local and global aspects. We propose an
attention refinement unit paired with local variance loss to impose supervision
on the attention model at each decoding step, and a global variance loss to
optimize the attention distributions of all decoding steps from the global
perspective. The performances on the CNN/Daily Mail dataset verify the
effectiveness of our methods.
