  Past work on story generation has demonstrated the usefulness of conditioning
on a generation plan to generate coherent stories. However, these approaches
have used heuristics or off-the-shelf models to first tag training stories with
the desired type of plan, and then train generation models in a supervised
fashion. In this paper, we propose a deep latent variable model that first
samples a sequence of anchor words, one per sentence in the story, as part of
its generative process. During training, our model treats the sequence of
anchor words as a latent variable and attempts to induce anchoring sequences
that help guide generation in an unsupervised fashion. We conduct experiments
with several types of sentence decoder distributions: left-to-right and
non-monotonic, with different degrees of restriction. Further, since we use
amortized variational inference to train our model, we introduce two
corresponding types of inference network for predicting the posterior on anchor
words. We conduct human evaluations which demonstrate that the stories produced
by our model are rated better in comparison with baselines which do not
consider story plans, and are similar or better in quality relative to
baselines which use external supervision for plans. Additionally, the proposed
model gets favorable scores when evaluated on perplexity, diversity, and
control of story via discrete plan.
