  Commonsense question answering has demonstrated considerable potential across
various applications like assistants and social robots. Although fully
fine-tuned pre-trained Language Models(LM) have achieved remarkable performance
in commonsense reasoning, their tendency to excessively prioritize textual
information hampers the precise transfer of structural knowledge and undermines
interpretability. Some studies have explored combining LMs with Knowledge
Graphs(KGs) by coarsely fusing the two modalities to perform Graph Neural
Network(GNN)-based reasoning that lacks a profound interaction between
heterogeneous modalities. In this paper, we propose a novel Graph-based
Structure-Aware Prompt Learning Model for commonsense reasoning, named G-SAP,
aiming to maintain a balance between heterogeneous knowledge and enhance the
cross-modal interaction within the LM+GNNs model. In particular, an evidence
graph is constructed by integrating multiple knowledge sources, i.e.
ConceptNet, Wikipedia, and Cambridge Dictionary to boost the performance.
Afterward, a structure-aware frozen PLM is employed to fully incorporate the
structured and textual information from the evidence graph, where the
generation of prompts is driven by graph entities and relations. Finally, a
heterogeneous message-passing reasoning module is used to facilitate deep
interaction of knowledge between the LM and graph-based networks. Empirical
validation, conducted through extensive experiments on three benchmark
datasets, demonstrates the notable performance of the proposed model. The
results reveal a significant advancement over the existing models, especially,
with 6.12% improvement over the SoTA LM+GNNs model on the OpenbookQA dataset.
