  Teaching neural models to generate narrative coherent texts is a critical
problem. Recent pre-trained language models have achieved promising results,
but there is still a gap between human written texts and machine-generated
outputs. In this work, we propose a novel multi-task training strategy for
coherent text generation grounded on the cognitive theory of writing, which
empowers the model to learn essential subskills needed for writing including
planning and reviewing besides end-to-end generation. We extensively evaluate
our model on three open-ended generation tasks including story generation, news
article writing and argument generation. Experiments show that our model
achieves better results on both few-shot and fully-supervised settings than
strong baselines, and human evaluations confirm that our model can generate
more coherent outputs.
