  Data owners face increasing liability for how the use of their data could
harm under-priviliged communities. Stakeholders would like to identify the
characteristics of data that lead to algorithms being biased against any
particular demographic groups, for example, defined by their race, gender, age,
and/or religion. Specifically, we are interested in identifying subsets of the
feature space where the ground truth response function from features to
observed outcomes differs across demographic groups. To this end, we propose
FORESEE, a FORESt of decision trEEs algorithm, which generates a score that
captures how likely an individual's response varies with sensitive attributes.
Empirically, we find that our approach allows us to identify the individuals
who are most likely to be misclassified by several classifiers, including
Random Forest, Logistic Regression, Support Vector Machine, and k-Nearest
Neighbors. The advantage of our approach is that it allows stakeholders to
characterize risky samples that may contribute to discrimination, as well as,
use the FORESEE to estimate the risk of upcoming samples.
