  Interpretability provides a means for humans to verify aspects of machine
learning (ML) models and empower human+ML teaming in situations where the task
cannot be fully automated. Different contexts require explanations with
different properties. For example, the kind of explanation required to
determine if an early cardiac arrest warning system is ready to be integrated
into a care setting is very different from the type of explanation required for
a loan applicant to help determine the actions they might need to take to make
their application successful.
  Unfortunately, there is a lack of standardization when it comes to properties
of explanations: different papers may use the same term to mean different
quantities, and different terms to mean the same quantity. This lack of a
standardized terminology and categorization of the properties of ML
explanations prevents us from both rigorously comparing interpretable machine
learning methods and identifying what properties are needed in what contexts.
  In this work, we survey properties defined in interpretable machine learning
papers, synthesize them based on what they actually measure, and describe the
trade-offs between different formulations of these properties. In doing so, we
enable more informed selection of task-appropriate formulations of explanation
properties as well as standardization for future work in interpretable machine
learning.
