  We explore story generation: creative systems that can build coherent and
fluent passages of text about a topic. We collect a large dataset of 300K
human-written stories paired with writing prompts from an online forum. Our
dataset enables hierarchical story generation, where the model first generates
a premise, and then transforms it into a passage of text. We gain further
improvements with a novel form of model fusion that improves the relevance of
the story to the prompt, and adding a new gated multi-scale self-attention
mechanism to model long-range context. Experiments show large improvements over
strong baselines on both automated and human evaluations. Human judges prefer
stories generated by our approach to those from a strong non-hierarchical model
by a factor of two to one.
