  Recent years have witnessed an increasing interest in training machines with
reasoning ability, which deeply relies on accurately and clearly presented clue
forms. The clues are usually modeled as entity-aware knowledge in existing
studies. However, those entity-aware clues are primarily focused on
commonsense, making them insufficient for tasks that require knowledge of
temporary facts or events, particularly in logical reasoning for reading
comprehension. To address this challenge, we are motivated to cover both
commonsense and temporary knowledge clues hierarchically. Specifically, we
propose a general formalism of knowledge units by extracting backbone
constituents of the sentence, such as the subject-verb-object formed ``facts''.
We then construct a supergraph on top of the fact units, allowing for the
benefit of sentence-level (relations among fact groups) and entity-level
interactions (concepts or actions inside a fact). Experimental results on
logical reasoning benchmarks and dialogue modeling datasets show that our
approach improves the baselines substantially, and it is general across
backbone models. Code is available at
\url{https://github.com/ozyyshr/FocalReasoner}.
