  This paper presents a new adversarial learning method for generative
conversational agents (GCA) besides a new model of GCA. Similar to previous
works on adversarial learning for dialogue generation, our method assumes the
GCA as a generator that aims at fooling a discriminator that labels dialogues
as human-generated or machine-generated; however, in our approach, the
discriminator performs token-level classification, i.e. it indicates whether
the current token was generated by humans or machines. To do so, the
discriminator also receives the context utterances (the dialogue history) and
the incomplete answer up to the current token as input. This new approach makes
possible the end-to-end training by backpropagation. A self-conversation
process enables to produce a set of generated data with more diversity for the
adversarial training. This approach improves the performance on questions not
related to the training data. Experimental results with human and adversarial
evaluations show that the adversarial method yields significant performance
gains over the usual teacher forcing training.
