  This paper presents new theory and methodology for the Bayesian estimation of
overfitted hidden Markov models, with finite state space. The goal is then to
achieve posterior emptying of extra states. A prior configuration is
constructed which favours configurations where the hidden Markov chain remains
ergodic although it empties out some of the states. Asymptotic posterior
convergence rates are proven theoretically, and demonstrated with a large
sample simulation. The problem of overfitted HMMs is then considered in the
context of smaller sample sizes, and due to computational and mixing issues two
alternative prior structures are studied, one commonly used in practice, and a
mixture of the two priors. The Prior Parallel Tempering approach of van Havre
(2015) is also extended to HMMs to allow MCMC estimation of the complex
posterior space. A replicate simulation study and an in-depth exploration is
performed to compare the three priors with hyperparameters chosen according to
the asymptotic constraints alongside less informative alternatives.
