  Much is now known about the consistency of Bayesian updating on
infinite-dimensional parameter spaces with independent or Markovian data.
Necessary conditions for consistency include the prior putting enough weight on
the correct neighborhoods of the data-generating distribution; various
sufficient conditions further restrict the prior in ways analogous to capacity
control in frequentist nonparametrics. The asymptotics of Bayesian updating
with mis-specified models or priors, or non-Markovian data, are far less well
explored. Here I establish sufficient conditions for posterior convergence when
all hypotheses are wrong, and the data have complex dependencies. The main
dynamical assumption is the asymptotic equipartition (Shannon-McMillan-Breiman)
property of information theory. This, along with Egorov's Theorem on uniform
convergence, lets me build a sieve-like structure for the prior. The main
statistical assumption, also a form of capacity control, concerns the
compatibility of the prior and the data-generating process, controlling the
fluctuations in the log-likelihood when averaged over the sieve-like sets. In
addition to posterior convergence, I derive a kind of large deviations
principle for the posterior measure, extending in some cases to rates of
convergence, and discuss the advantages of predicting using a combination of
models known to be wrong. An appendix sketches connections between these
results and the replicator dynamics of evolutionary theory.
