  We present a system for answering questions based on the full text of books
(BookQA), which first selects book passages given a question at hand, and then
uses a memory network to reason and predict an answer. To improve
generalization, we pretrain our memory network using artificial questions
generated from book sentences. We experiment with the recently published
NarrativeQA corpus, on the subset of Who questions, which expect book
characters as answers. We experimentally show that BERT-based retrieval and
pretraining improve over baseline results significantly. At the same time, we
confirm that NarrativeQA is a highly challenging data set, and that there is
need for novel research in order to achieve high-precision BookQA results. We
analyze some of the bottlenecks of the current approach, and we argue that more
research is needed on text representation, retrieval of relevant passages, and
reasoning, including commonsense knowledge.
