  Despite the popularity of Shapley Values in explaining neural text
classification models, computing them is prohibitive for large pretrained
models due to a large number of model evaluations. In practice, Shapley Values
are often estimated with a small number of stochastic model evaluations.
However, we show that the estimated Shapley Values are sensitive to random seed
choices -- the top-ranked features often have little overlap across different
seeds, especially on examples with longer input texts. This can only be
mitigated by aggregating thousands of model evaluations, which on the other
hand, induces substantial computational overheads. To mitigate the trade-off
between stability and efficiency, we develop an amortized model that directly
predicts each input feature's Shapley Value without additional model
evaluations. It is trained on a set of examples whose Shapley Values are
estimated from a large number of model evaluations to ensure stability.
Experimental results on two text classification datasets demonstrate that our
amortized model estimates Shapley Values accurately with up to 60 times speedup
compared to traditional methods. Furthermore, the estimated values are stable
as the inference is deterministic. We release our code at
https://github.com/yangalan123/Amortized-Interpretability.
