  We consider importance sampling (IS) type weighted estimators based on Markov
chain Monte Carlo (MCMC) targeting an approximate marginal of the target
distribution. In the context of Bayesian latent variable models, the MCMC
typically operates on the hyperparameters, and the subsequent weighting may be
based on IS or sequential Monte Carlo (SMC), but allows for multilevel
techniques as well. The IS approach provides a natural alternative to delayed
acceptance (DA) pseudo-marginal/particle MCMC, and has many advantages over DA,
including a straightforward parallelisation and additional flexibility in MCMC
implementation. We detail minimal conditions which ensure strong consistency of
the suggested estimators, and provide central limit theorems with expressions
for asymptotic variances. We demonstrate how our method can make use of SMC in
the state space models context, using Laplace approximations and
time-discretised diffusions. Our experimental results are promising and show
that the IS type approach can provide substantial gains relative to an
analogous DA scheme, and is often competitive even without parallelisation.
