  Responsible use of machine learning requires models to be audited for
undesirable properties. While a body of work has proposed using explanations
for auditing, how to do so and why has remained relatively ill-understood. This
work formalizes the role of explanations in auditing and investigates if and
how model explanations can help audits. Specifically, we propose
explanation-based algorithms for auditing linear classifiers and decision trees
for feature sensitivity. Our results illustrate that Counterfactual
explanations are extremely helpful for auditing. While Anchors and decision
paths may not be as beneficial in the worst-case, in the average-case they do
aid a lot.
