  Model interpretations are often used in practice to extract real world
insights from machine learning models. These interpretations have a wide range
of applications; they can be presented as business recommendations or used to
evaluate model bias. It is vital for a data scientist to choose trustworthy
interpretations to drive real world impact. Doing so requires an understanding
of how the accuracy of a model impacts the quality of standard interpretation
tools. In this paper, we will explore how a model's predictive accuracy affects
interpretation quality. We propose two metrics to quantify the quality of an
interpretation and design an experiment to test how these metrics vary with
model accuracy. We find that for datasets that can be modeled accurately by a
variety of methods, simpler methods yield higher quality interpretations. We
also identify which interpretation method works the best for lower levels of
model accuracy.
