  Recently Graph Neural Network (GNN) has been applied successfully to various
NLP tasks that require reasoning, such as multi-hop machine reading
comprehension. In this paper, we consider a novel case where reasoning is
needed over graphs built from sequences, i.e. graph nodes with sequence data.
Existing GNN models fulfill this goal by first summarizing the node sequences
into fixed-dimensional vectors, then applying GNN on these vectors. To avoid
information loss inherent in the early summarization and make sequential
labeling tasks on GNN output feasible, we propose a new type of GNN called
Graph Sequential Network (GSN), which features a new message passing algorithm
based on co-attention between a node and each of its neighbors. We validate the
proposed GSN on two NLP tasks: interpretable multi-hop reading comprehension on
HotpotQA and graph based fact verification on FEVER. Both tasks require
reasoning over multiple documents or sentences. Our experimental results show
that the proposed GSN attains better performance than the standard GNN based
methods.
