  In recent years researchers have achieved considerable success applying
neural network methods to question answering (QA). These approaches have
achieved state of the art results in simplified closed-domain settings such as
the SQuAD (Rajpurkar et al., 2016) dataset, which provides a pre-selected
passage, from which the answer to a given question may be extracted. More
recently, researchers have begun to tackle open-domain QA, in which the model
is given a question and access to a large corpus (e.g., wikipedia) instead of a
pre-selected passage (Chen et al., 2017a). This setting is more complex as it
requires large-scale search for relevant passages by an information retrieval
component, combined with a reading comprehension model that "reads" the
passages to generate an answer to the question. Performance in this setting
lags considerably behind closed-domain performance. In this paper, we present a
novel open-domain QA system called Reinforced Ranker-Reader $(R^3)$, based on
two algorithmic innovations. First, we propose a new pipeline for open-domain
QA with a Ranker component, which learns to rank retrieved passages in terms of
likelihood of generating the ground-truth answer to a given question. Second,
we propose a novel method that jointly trains the Ranker along with an
answer-generation Reader model, based on reinforcement learning. We report
extensive experimental results showing that our method significantly improves
on the state of the art for multiple open-domain QA datasets.
