  This paper deals with the ergodicity and the existence of a strong law of
large numbers for adaptive Markov Chain Monte Carlo. We show that a diminishing
adaptation assumption together with a drift condition for positive recurrence
is enough to imply ergodicity. Strengthening the drift condition to a
polynomial drift condition yields a strong law of large numbers for possibly
unbounded functions. These results broaden considerably the class of adaptive
MCMC algorithms for which rigorous analysis is now possible. As an example, we
give a detailed analysis of the Adaptive Metropolis Algorithm of Haario et al.
(2001) when the target distribution is sub-exponential in the tails.
