  As a solution concept in cooperative game theory, Shapley value is highly
recognized in model interpretability studies and widely adopted by the leading
Machine Learning as a Service (MLaaS) providers, such as Google, Microsoft, and
IBM. However, as the Shapley value-based model interpretability methods have
been thoroughly studied, few researchers consider the privacy risks incurred by
Shapley values, despite that interpretability and privacy are two foundations
of machine learning (ML) models.
  In this paper, we investigate the privacy risks of Shapley value-based model
interpretability methods using feature inference attacks: reconstructing the
private model inputs based on their Shapley value explanations. Specifically,
we present two adversaries. The first adversary can reconstruct the private
inputs by training an attack model based on an auxiliary dataset and black-box
access to the model interpretability services. The second adversary, even
without any background knowledge, can successfully reconstruct most of the
private features by exploiting the local linear correlations between the model
inputs and outputs. We perform the proposed attacks on the leading MLaaS
platforms, i.e., Google Cloud, Microsoft Azure, and IBM aix360. The
experimental results demonstrate the vulnerability of the state-of-the-art
Shapley value-based model interpretability methods used in the leading MLaaS
platforms and highlight the significance and necessity of designing
privacy-preserving model interpretability methods in future studies. To our
best knowledge, this is also the first work that investigates the privacy risks
of Shapley values.
