  Exact computation of various machine learning explanations requires numerous
model evaluations and in extreme cases becomes impractical. The computational
cost of approximation increases with an ever-increasing size of data and model
parameters. Many heuristics have been proposed to approximate post-hoc
explanations efficiently. This paper shows that the standard i.i.d. sampling
used in a broad spectrum of algorithms for explanation estimation leads to an
approximation error worthy of improvement. To this end, we introduce Compress
Then Explain (CTE), a new paradigm for more efficient and accurate explanation
estimation. CTE uses distribution compression through kernel thinning to obtain
a data sample that best approximates the marginal distribution. We show that
CTE improves the estimation of removal-based local and global explanations with
negligible computational overhead. It often achieves an on-par explanation
approximation error using 2-3x less samples, i.e. requiring 2-3x less model
evaluations. CTE is a simple, yet powerful, plug-in for any explanation method
that now relies on i.i.d. sampling.
