  This paper explores the task of answer-aware questions generation. Based on
the attention-based pointer generator model, we propose to incorporate an
auxiliary task of language modeling to help question generation in a
hierarchical multi-task learning structure. Our joint-learning model enables
the encoder to learn a better representation of the input sequence, which will
guide the decoder to generate more coherent and fluent questions. On both SQuAD
and MARCO datasets, our multi-task learning model boosts the performance,
achieving state-of-the-art results. Moreover, human evaluation further proves
the high quality of our generated questions.
