  Across various sectors such as healthcare, criminal justice, national
security, finance, and technology, large-scale machine learning (ML) and
artificial intelligence (AI) systems are being deployed to make critical
data-driven decisions. Many have asked if we can and should trust these ML
systems to be making these decisions. Two critical components are prerequisites
for trust in ML systems: interpretability, or the ability to understand why the
ML system makes the decisions it does, and fairness, which ensures that ML
systems do not exhibit bias against certain individuals or groups. Both
interpretability and fairness are important and have separately received
abundant attention in the ML literature, but so far, there have been very few
methods developed to directly interpret models with regard to their fairness.
In this paper, we focus on arguably the most popular type of ML interpretation:
feature importance scores. Inspired by the use of decision trees in knowledge
distillation, we propose to leverage trees as interpretable surrogates for
complex black-box ML models. Specifically, we develop a novel fair feature
importance score for trees that can be used to interpret how each feature
contributes to fairness or bias in trees, tree-based ensembles, or tree-based
surrogates of any complex ML system. Like the popular mean decrease in impurity
for trees, our Fair Feature Importance Score is defined based on the mean
decrease (or increase) in group bias. Through simulations as well as real
examples on benchmark fairness datasets, we demonstrate that our Fair Feature
Importance Score offers valid interpretations for both tree-based ensembles and
tree-based surrogates of other ML systems.
