  We introduce region-based explanations (RbX), a novel, model-agnostic method
to generate local explanations of scalar outputs from a black-box prediction
model using only query access. RbX is based on a greedy algorithm for building
a convex polytope that approximates a region of feature space where model
predictions are close to the prediction at some target point. This region is
fully specified by the user on the scale of the predictions, rather than on the
scale of the features. The geometry of this polytope - specifically the change
in each coordinate necessary to escape the polytope - quantifies the local
sensitivity of the predictions to each of the features. These "escape
distances" can then be standardized to rank the features by local importance.
RbX is guaranteed to satisfy a "sparsity axiom," which requires that features
which do not enter into the prediction model are assigned zero importance. At
the same time, real data examples and synthetic experiments show how RbX can
more readily detect all locally relevant features than existing methods.
