  There has been a surge of interest in utilizing Knowledge Graphs (KGs) for
various natural language processing/understanding tasks. The conventional
mechanism to retrieve facts in KGs usually involves three steps: entity span
detection, entity disambiguation, and relation classification. However, this
approach requires additional labels for training each of the three
subcomponents in addition to pairs of input texts and facts, and also may
accumulate errors propagated from failures in previous steps. To tackle these
limitations, we propose a simple knowledge retrieval framework, which directly
retrieves facts from the KGs given the input text based on their
representational similarities, which we refer to as Direct Fact Retrieval
(DiFaR). Specifically, we first embed all facts in KGs onto a dense embedding
space by using a language model trained by only pairs of input texts and facts,
and then provide the nearest facts in response to the input text. Since the
fact, consisting of only two entities and one relation, has little context to
encode, we propose to further refine ranks of top-k retrieved facts with a
reranker that contextualizes the input text and the fact jointly. We validate
our DiFaR framework on multiple fact retrieval tasks, showing that it
significantly outperforms relevant baselines that use the three-step approach.
