  General Question Answering (QA) systems over texts require the multi-hop
reasoning capability, i.e. the ability to reason with information collected
from multiple passages to derive the answer. In this paper we conduct a
systematic analysis to assess such an ability of various existing models
proposed for multi-hop QA tasks. Specifically, our analysis investigates that
whether providing the full reasoning chain of multiple passages, instead of
just one final passage where the answer appears, could improve the performance
of the existing QA models. Surprisingly, when using the additional evidence
passages, the improvements of all the existing multi-hop reading approaches are
rather limited, with the highest error reduction of 5.8% on F1 (corresponding
to 1.3% absolute improvement) from the BERT model.
  To better understand whether the reasoning chains could indeed help find
correct answers, we further develop a co-matching-based method that leads to
13.1% error reduction with passage chains when applied to two of our base
readers (including BERT). Our results demonstrate the existence of the
potential improvement using explicit multi-hop reasoning and the necessity to
develop models with better reasoning abilities.
