  Markov chain Monte Carlo (MCMC) methods are frequently used to approximately
simulate high-dimensional, multimodal probability distributions. In adaptive
MCMC methods, the transition kernel is changed "on the fly" in the hope to
speed up convergence. We study interacting tempering, an adaptive MCMC
algorithm based on interacting Markov chains, that can be seen as a simplified
version of the equi-energy sampler. Using a coupling argument, we show that
under easy to verify assumptions on the target distribution (on a finite
space), the interacting tempering process rapidly forgets its starting
distribution. The result applies, among others, to exponential random graph
models, the Ising and Potts models (in mean field or on a bounded degree
graph), as well as (Edwards-Anderson) Ising spin glasses. As a cautionary note,
we also exhibit an example of a target distribution for which the interacting
tempering process rapidly forgets its starting distribution, but takes an
exponential number of steps (in the dimension of the state space) to converge
to its limiting distribution. As a consequence, we argue that convergence
diagnostics that are based on demonstrating that the process has forgotten its
starting distribution might be of limited use for adaptive MCMC algorithms like
interacting tempering.
