  We provide a general methodology for unbiased estimation for intractable
stochastic models. We consider situations where the target distribution can be
written as an appropriate limit of distributions, and where conventional
approaches require truncation of such a representation leading to a systematic
bias. For example, the target distribution might be representable as the
$L^2$-limit of a basis expansion in a suitable Hilbert space; or alternatively
the distribution of interest might be representable as the weak limit of a
sequence of random variables, as in MCMC. Our main motivation comes from
infinite-dimensional models which can be parame- terised in terms of a series
expansion of basis functions (such as that given by a Karhunen-Loeve
expansion). We consider schemes for direct unbiased estimation along such an
expansion, as well as those based on MCMC schemes which, due to their
dimensionality, cannot be directly imple- mented, but which can be effectively
estimated unbiasedly. For all our methods we give theory to justify the
numerical stability for robust Monte Carlo implementation, and in some cases we
illustrate using simulations. Interestingly the computational efficiency of our
methods is usually comparable to simpler methods which are biased. Crucial to
the effectiveness of our proposed methodology is the construction of
appropriate couplings, many of which resonate strongly with the Monte Carlo
constructions used in the coupling from the past algorithm and its variants.
