  We introduce the novel task of answering entity-seeking recommendation
questions using a collection of reviews that describe candidate answer
entities. We harvest a QA dataset that contains 47,124 paragraph-sized real
user questions from travelers seeking recommendations for hotels, attractions
and restaurants. Each question can have thousands of candidate answers to
choose from and each candidate is associated with a collection of unstructured
reviews. This dataset is especially challenging because commonly used neural
architectures for reasoning and QA are prohibitively expensive for a task of
this scale. As a solution, we design a scalable cluster-select-rerank approach.
It first clusters text for each entity to identify exemplar sentences
describing an entity. It then uses a scalable neural information retrieval (IR)
module to select a set of potential entities from the large candidate set. A
reranker uses a deeper attention-based architecture to pick the best answers
from the selected entities. This strategy performs better than a pure IR or a
pure attention-based reasoning approach yielding nearly 25% relative
improvement in Accuracy@3 over both approaches.
