  Supervised Machine Learning (SML) algorithms such as Gradient Boosting,
Random Forest, and Neural Networks have become popular in recent years due to
their increased predictive performance over traditional statistical methods.
This is especially true with large data sets (millions or more observations and
hundreds to thousands of predictors). However, the complexity of the SML models
makes them opaque and hard to interpret without additional tools. There has
been a lot of interest recently in developing global and local diagnostics for
interpreting and explaining SML models. In this paper, we propose locally
interpretable models and effects based on supervised partitioning (trees)
referred to as LIME-SUP. This is in contrast with the KLIME approach that is
based on clustering the predictor space. We describe LIME-SUP based on fitting
trees to the fitted response (LIM-SUP-R) as well as the derivatives of the
fitted response (LIME-SUP-D). We compare the results with KLIME and describe
its advantages using simulation and real data.
