  Recent work has proposed multi-hop models and datasets for studying complex
natural language reasoning. One notable task requiring multi-hop reasoning is
fact checking, where a set of connected evidence pieces leads to the final
verdict of a claim. However, existing datasets either do not provide
annotations for gold evidence pages, or the only dataset which does (FEVER)
mostly consists of claims which can be fact-checked with simple reasoning and
is constructed artificially. Here, we study more complex claim verification of
naturally occurring claims with multiple hops over interconnected evidence
chunks. We: 1) construct a small annotated dataset, PolitiHop, of evidence
sentences for claim verification; 2) compare it to existing multi-hop datasets;
and 3) study how to transfer knowledge from more extensive in- and
out-of-domain resources to PolitiHop. We find that the task is complex and
achieve the best performance with an architecture that specifically models
reasoning over evidence pieces in combination with in-domain transfer learning.
