  The rise of AI methods to make predictions and decisions has led to a
pressing need for more explainable artificial intelligence (XAI) methods. One
common approach for XAI is to produce a post-hoc explanation, explaining why a
black box ML model made a certain prediction. Formal approaches to post-hoc
explanations provide succinct reasons for why a prediction was made, as well as
why not another prediction was made. But these approaches assume that features
are independent and uniformly distributed. While this means that "why"
explanations are correct, they may be longer than required. It also means the
"why not" explanations may be suspect as the counterexamples they rely on may
not be meaningful. In this paper, we show how one can apply background
knowledge to give more succinct "why" formal explanations, that are presumably
easier to interpret by humans, and give more accurate "why not" explanations.
In addition, we show how to use existing rule induction techniques to
efficiently extract background information from a dataset, and also how to
report which background information was used to make an explanation, allowing a
human to examine it if they doubt the correctness of the explanation.
