  The broad adoption of Machine Learning (ML) in security-critical fields
demands the explainability of the approach. However, the research on
understanding ML models, such as Random Forest (RF), is still in its infant
stage. In this work, we leverage formal methods and logical reasoning to
develop a novel model-specific method for explaining the prediction of RF. Our
approach is centered around Minimal Unsatisfiable Cores (MUC) and provides a
comprehensive solution for feature importance, covering local and global
aspects, and adversarial sample analysis. Experimental results on several
datasets illustrate the high quality of our feature importance measurement. We
also demonstrate that our adversarial analysis outperforms the state-of-the-art
method. Moreover, our method can produce a user-centered report, which helps
provide recommendations in real-life applications.
