  With larger data at their disposal, scientists are emboldened to tackle
complex questions that require sophisticated statistical models. It is not
unusual for the latter to have likelihood functions that elude analytical
formulations. Even under such adversity, when one can simulate from the
sampling distribution, Bayesian analysis can be conducted using approximate
methods such as Approximate Bayesian Computation (ABC) or Bayesian Synthetic
Likelihood (BSL). A significant drawback of these methods is that the number of
required simulations can be prohibitively large, thus severely limiting their
scope. In this paper we design perturbed MCMC samplers that can be used within
the ABC and BSL paradigms to significantly accelerate computation while
maintaining control on computational efficiency. The proposed strategy relies
on recycling samples from the chain's past. The algorithmic design is supported
by a theoretical analysis while practical performance is examined via a series
of simulation examples and data analyses.
