  Group fairness metrics can detect when a deep learning model behaves
differently for advantaged and disadvantaged groups, but even models that score
well on these metrics can make blatantly unfair predictions. We present smooth
prediction sensitivity, an efficiently computed measure of individual fairness
for deep learning models that is inspired by ideas from interpretability in
deep learning. smooth prediction sensitivity allows individual predictions to
be audited for fairness. We present preliminary experimental results suggesting
that smooth prediction sensitivity can help distinguish between fair and unfair
predictions, and that it may be helpful in detecting blatantly unfair
predictions from "group-fair" models.
