  Question Answering (QA), as a research field, has primarily focused on either
knowledge bases (KBs) or free text as a source of knowledge. These two sources
have historically shaped the kinds of questions that are asked over these
sources, and the methods developed to answer them. In this work, we look
towards a practical use-case of QA over user-instructed knowledge that uniquely
combines elements of both structured QA over knowledge bases, and unstructured
QA over narrative, introducing the task of multi-relational QA over personal
narrative. As a first step towards this goal, we make three key contributions:
(i) we generate and release TextWorldsQA, a set of five diverse datasets, where
each dataset contains dynamic narrative that describes entities and relations
in a simulated world, paired with variably compositional questions over that
knowledge, (ii) we perform a thorough evaluation and analysis of several
state-of-the-art QA models and their variants at this task, and (iii) we
release a lightweight Python-based framework we call TextWorlds for easily
generating arbitrary additional worlds and narrative, with the goal of allowing
the community to create and share a growing collection of diverse worlds as a
test-bed for this task.
