  Existing LLM-based systems for writing long-form stories or story outlines
frequently suffer from unnatural pacing, whether glossing over important events
or over-elaborating on insignificant details, resulting in a jarring experience
for the reader. We propose a CONCrete Outline ConTrol (CONCOCT) system to
improve pacing when automatically generating story outlines. We first train a
concreteness evaluator to judge which of two events is more concrete
(low-level-detailed). This evaluator can then be used to control pacing in
hierarchical outline generation; in this work, we explore a vaguest-first
expansion procedure that aims for uniform pacing. We further use the evaluator
to filter new outline items based on predicted concreteness. Compared to a
baseline hierarchical outline generator, humans judge CONCOCT's pacing to be
more consistent over 57% of the time across multiple outline lengths; the gains
also translate to downstream stories. All code, data, and models are
open-sourced.
