  In healthcare, making the best possible predictions with complex models
(e.g., neural networks, ensembles/stacks of different models) can impact
patient welfare. In order to make these complex models explainable, we present
DeepSHAP for mixed model types, a framework for layer wise propagation of
Shapley values that builds upon DeepLIFT (an existing approach for explaining
neural networks). We show that in addition to being able to explain neural
networks, this new framework naturally enables attributions for stacks of mixed
models (e.g., neural network feature extractor into a tree model) as well as
attributions of the loss. Finally, we theoretically justify a method for
obtaining attributions with respect to a background distribution (under a
Shapley value framework).
