  Rule-based classification models described in the language of logic directly
predict boolean values, rather than modeling a probability and translating it
into a prediction as done in statistical models. The vast majority of existing
uncertainty quantification approaches rely on models providing continuous
output not available to rule-based models. In this work, we propose an
uncertainty quantification framework in the form of a meta-model that takes any
binary classifier with binary output as a black box and estimates the
prediction accuracy of that base model at a given input along with a level of
confidence on that estimation. The confidence is based on how well that input
region is explored and is designed to work in any OOD scenario. We demonstrate
the usefulness of this uncertainty model by building an abstaining classifier
powered by it and observing its performance in various scenarios.
