  Shapley values provide model agnostic feature attributions for model outcome
at a particular instance by simulating feature absence under a global
population distribution. The use of a global population can lead to potentially
misleading results when local model behaviour is of interest. Hence we consider
the formulation of neighbourhood reference distributions that improve the local
interpretability of Shapley values. By doing so, we find that the
Nadaraya-Watson estimator, a well-studied kernel regressor, can be expressed as
a self-normalised importance sampling estimator. Empirically, we observe that
Neighbourhood Shapley values identify meaningful sparse feature relevance
attributions that provide insight into local model behaviour, complimenting
conventional Shapley analysis. They also increase on-manifold explainability
and robustness to the construction of adversarial classifiers.
