  Recent work in model-agnostic explanations of black-box machine learning has
demonstrated that interpretability of complex models does not have to come at
the cost of accuracy or model flexibility. However, it is not clear what kind
of explanations, such as linear models, decision trees, and rule lists, are the
appropriate family to consider, and different tasks and models may benefit from
different kinds of explanations. Instead of picking a single family of
representations, in this work we propose to use "programs" as model-agnostic
explanations. We show that small programs can be expressive yet intuitive as
explanations, and generalize over a number of existing interpretable families.
We propose a prototype program induction method based on simulated annealing
that approximates the local behavior of black-box classifiers around a specific
prediction using random perturbations. Finally, we present preliminary
application on small datasets and show that the generated explanations are
intuitive and accurate for a number of classifiers.
