  Influence functions (IFs) are a powerful tool for detecting anomalous
examples in large scale datasets. However, they are unstable when applied to
deep networks. In this paper, we provide an explanation for the instability of
IFs and develop a solution to this problem. We show that IFs are unreliable
when the two data points belong to two different classes. Our solution
leverages class information to improve the stability of IFs. Extensive
experiments show that our modification significantly improves the performance
and stability of IFs while incurring no additional computational cost.
