  Answering complex questions often requires reasoning over knowledge graphs
(KGs). State-of-the-art methods often utilize entities in questions to retrieve
local subgraphs, which are then fed into KG encoder, e.g. graph neural networks
(GNNs), to model their local structures and integrated into language models for
question answering. However, this paradigm constrains retrieved knowledge in
local subgraphs and discards more diverse triplets buried in KGs that are
disconnected but useful for question answering. In this paper, we propose a
simple yet effective method to first retrieve the most relevant triplets from
KGs and then rerank them, which are then concatenated with questions to be fed
into language models. Extensive results on both CommonsenseQA and OpenbookQA
datasets show that our method can outperform state-of-the-art up to 4.6%
absolute accuracy.
