  The natural language generation (NLG) module in a task-oriented dialogue
system produces user-facing utterances conveying required information. Thus, it
is critical for the generated response to be natural and fluent. We propose to
integrate adversarial training to produce more human-like responses. The model
uses Straight-Through Gumbel-Softmax estimator for gradient computation. We
also propose a two-stage training scheme to boost performance. Empirical
results show that the adversarial training can effectively improve the quality
of language generation in both automatic and human evaluations. For example, in
the RNN-LG Restaurant dataset, our model AdvNLG outperforms the previous
state-of-the-art result by 3.6% in BLEU.
