  Understanding stories is a challenging reading comprehension problem for
machines as it requires reading a large volume of text and following long-range
dependencies. In this paper, we introduce the Shmoop Corpus: a dataset of 231
stories that are paired with detailed multi-paragraph summaries for each
individual chapter (7,234 chapters), where the summary is chronologically
aligned with respect to the story chapter. From the corpus, we construct a set
of common NLP tasks, including Cloze-form question answering and a simplified
form of abstractive summarization, as benchmarks for reading comprehension on
stories. We then show that the chronological alignment provides a strong
supervisory signal that learning-based methods can exploit leading to
significant improvements on these tasks. We believe that the unique structure
of this corpus provides an important foothold towards making machine story
comprehension more approachable.
