  Transformer, based on the encoder-decoder framework, has achieved
state-of-the-art performance on several natural language generation tasks. The
encoder maps the words in the input sentence into a sequence of hidden states,
which are then fed into the decoder to generate the output sentence. These
hidden states usually correspond to the input words and focus on capturing
local information. However, the global (sentence level) information is seldom
explored, leaving room for the improvement of generation quality. In this
paper, we propose a novel global representation enhanced Transformer (GRET) to
explicitly model global representation in the Transformer network.
Specifically, in the proposed model, an external state is generated for the
global representation from the encoder. The global representation is then fused
into the decoder during the decoding process to improve generation quality. We
conduct experiments in two text generation tasks: machine translation and text
summarization. Experimental results on four WMT machine translation tasks and
LCSTS text summarization task demonstrate the effectiveness of the proposed
approach on natural language generation.
