  Story generation is a challenging task, which demands to maintain consistency
of the plots and characters throughout the story. Previous works have shown
that GPT2, a large-scale language model, has achieved good performance on story
generation. However, we observe that several serious issues still exist in the
stories generated by GPT2 which can be categorized into two folds: consistency
and coherency. In terms of consistency, on one hand, GPT2 cannot guarantee the
consistency of the plots explicitly. On the other hand, the generated stories
usually contain coreference errors. In terms of coherency, GPT2 does not take
account of the discourse relations between sentences of stories directly. To
enhance the consistency and coherency of the generated stories, we propose a
two-stage generation framework, where the first stage is to organize the story
outline which depicts the story plots and events, and the second stage is to
expand the outline into a complete story. Therefore the plots consistency can
be controlled and guaranteed explicitly. In addition, coreference supervision
signals are incorporated to reduce coreference errors and improve the
coreference consistency. Moreover, we design an auxiliary task of discourse
relation modeling to improve the coherency of the generated stories.
Experimental results on a story dataset show that our model outperforms the
baseline approaches in terms of both automatic metrics and human evaluation.
