  Tree-based machine learning models such as random forests, decision trees,
and gradient boosted trees are the most popular non-linear predictive models
used in practice today, yet comparatively little attention has been paid to
explaining their predictions. Here we significantly improve the
interpretability of tree-based models through three main contributions: 1) The
first polynomial time algorithm to compute optimal explanations based on game
theory. 2) A new type of explanation that directly measures local feature
interaction effects. 3) A new set of tools for understanding global model
structure based on combining many local explanations of each prediction. We
apply these tools to three medical machine learning problems and show how
combining many high-quality local explanations allows us to represent global
structure while retaining local faithfulness to the original model. These tools
enable us to i) identify high magnitude but low frequency non-linear mortality
risk factors in the general US population, ii) highlight distinct population
sub-groups with shared risk characteristics, iii) identify non-linear
interaction effects among risk factors for chronic kidney disease, and iv)
monitor a machine learning model deployed in a hospital by identifying which
features are degrading the model's performance over time. Given the popularity
of tree-based machine learning models, these improvements to their
interpretability have implications across a broad set of domains.
