  In this paper we analyze two question answering tasks : the TREC-8 question
answering task and a set of reading comprehension exams. First, we show that
Q/A systems perform better when there are multiple answer opportunities per
question. Next, we analyze common approaches to two subproblems: term overlap
for answer sentence identification, and answer typing for short answer
extraction. We present general tools for analyzing the strengths and
limitations of techniques for these subproblems. Our results quantify the
limitations of both term overlap and answer typing to distinguish between
competing answer candidates.
