  Machine reading comprehension has aroused wide concerns, since it explores
the potential of model for text understanding. To further equip the machine
with the reasoning capability, the challenging task of logical reasoning is
proposed. Previous works on logical reasoning have proposed some strategies to
extract the logical units from different aspects. However, there still remains
a challenge to model the long distance dependency among the logical units.
Also, it is demanding to uncover the logical structures of the text and further
fuse the discrete logic to the continuous text embedding. To tackle the above
issues, we propose an end-to-end model Logiformer which utilizes a two-branch
graph transformer network for logical reasoning of text. Firstly, we introduce
different extraction strategies to split the text into two sets of logical
units, and construct the logical graph and the syntax graph respectively. The
logical graph models the causal relations for the logical branch while the
syntax graph captures the co-occurrence relations for the syntax branch.
Secondly, to model the long distance dependency, the node sequence from each
graph is fed into the fully connected graph transformer structures. The two
adjacent matrices are viewed as the attention biases for the graph transformer
layers, which map the discrete logical structures to the continuous text
embedding space. Thirdly, a dynamic gate mechanism and a question-aware
self-attention module are introduced before the answer prediction to update the
features. The reasoning process provides the interpretability by employing the
logical units, which are consistent with human cognition. The experimental
results show the superiority of our model, which outperforms the
state-of-the-art single model on two logical reasoning benchmarks.
