  Advanced large-scale neural language models have led to significant success
in many language generation tasks. However, the most commonly used training
objective, Maximum Likelihood Estimation (MLE), has been shown problematic,
where the trained model prefers using dull and repetitive phrases. In this
work, we introduce ScaleGrad, a modification straight to the gradient of the
loss function, to remedy the degeneration issue of the standard MLE objective.
By directly maneuvering the gradient information, ScaleGrad makes the model
learn to use novel tokens. Empirical results show the effectiveness of our
method not only in open-ended generation, but also in directed generation
tasks. With the simplicity in architecture, our method can serve as a general
training objective that is applicable to most of the neural text generation
tasks.
