  Stein importance sampling is a widely applicable technique based on
kernelized Stein discrepancy, which corrects the output of approximate sampling
algorithms by reweighting the empirical distribution of the samples. A general
analysis of this technique is conducted for the previously unconsidered setting
where samples are obtained via the simulation of a Markov chain, and applies to
an arbitrary underlying Polish space. We prove that Stein importance sampling
yields consistent estimators for quantities related to a target distribution of
interest by using samples obtained from a geometrically ergodic Markov chain
with a possibly unknown invariant measure that differs from the desired target.
The approach is shown to be valid under conditions that are satisfied for a
large number of unadjusted samplers, and is capable of retaining consistency
when data subsampling is used. Along the way, a universal theory of reproducing
Stein kernels is established, which enables the construction of kernelized
Stein discrepancy on general Polish spaces, and provides sufficient conditions
for kernels to be convergence-determining on such spaces. These results are of
independent interest for the development of future methodology based on
kernelized Stein discrepancies.
