  Commonsense reasoning in natural language is a desired ability of artificial
intelligent systems. For solving complex commonsense reasoning tasks, a typical
solution is to enhance pre-trained language models~(PTMs) with a
knowledge-aware graph neural network~(GNN) encoder that models a commonsense
knowledge graph~(CSKG). Despite the effectiveness, these approaches are built
on heavy architectures, and can't clearly explain how external knowledge
resources improve the reasoning capacity of PTMs. Considering this issue, we
conduct a deep empirical analysis, and find that it is indeed relation features
from CSKGs (but not node features) that mainly contribute to the performance
improvement of PTMs. Based on this finding, we design a simple MLP-based
knowledge encoder that utilizes statistical relation paths as features.
Extensive experiments conducted on five benchmarks demonstrate the
effectiveness of our approach, which also largely reduces the parameters for
encoding CSKGs. Our codes and data are publicly available at
https://github.com/RUCAIBox/SAFE.
