  For a reversible and ergodic Markov chain $\{X_n,n\geq0\}$ with invariant
distribution $\pi$, we show that a valid confidence interval for $\pi(h)$ can
be constructed whenever the asymptotic variance $\sigma^2_P(h)$ is finite and
positive. We do not impose any additional condition on the convergence rate of
the Markov chain. The confidence interval is derived using the so-called
fixed-b lag-window estimator of $\sigma_P^2(h)$. We also derive a result that
suggests that the proposed confidence interval procedure converges faster than
classical confidence interval procedures based on the Gaussian distribution and
standard central limit theorems for Markov chains.
