  We consider a time-average estimator $f_{k}$ of a functional of a Markov
chain. Under a coupling assumption, we show that the expectation of $f_{k}$ has
a limit $\mu$ as the number of time-steps goes to infinity. We describe a
modification of $f_{k}$ that yields an unbiased estimator $\hat f_{k}$ of
$\mu$. It is shown that $\hat f_{k}$ is square-integrable and has finite
expected running time. Under certain conditions, $\hat f_{k}$ can be built
without any precomputations, and is asymptotically at least as efficient as
$f_{k}$, up to a multiplicative constant arbitrarily close to $1$. Our approach
provides an unbiased estimator for the bias of $f_{k}$. We study applications
to volatility forecasting, queues, and the simulation of high-dimensional
Gaussian vectors. Our numerical experiments are consistent with our theoretical
findings.
