  Logical reasoning is of vital importance to natural language understanding.
Previous studies either employ graph-based models to incorporate prior
knowledge about logical relations, or introduce symbolic logic into neural
models through data augmentation. These methods, however, heavily depend on
annotated training data, and thus suffer from over-fitting and poor
generalization problems due to the dataset sparsity. To address these two
problems, in this paper, we propose MERIt, a MEta-path guided contrastive
learning method for logical ReasonIng of text, to perform self-supervised
pre-training on abundant unlabeled text data. Two novel strategies serve as
indispensable components of our method. In particular, a strategy based on
meta-path is devised to discover the logical structure in natural texts,
followed by a counterfactual data augmentation strategy to eliminate the
information shortcut induced by pre-training. The experimental results on two
challenging logical reasoning benchmarks, i.e., ReClor and LogiQA, demonstrate
that our method outperforms the SOTA baselines with significant improvements.
