  We investigate stationary hidden Markov processes for which mutual
information between the past and the future is infinite. It is assumed that the
number of observable states is finite and the number of hidden states is
countably infinite. Under this assumption, we show that the block mutual
information of a hidden Markov process is upper bounded by a power law
determined by the tail index of the hidden state distribution. Moreover, we
exhibit three examples of processes. The first example, considered previously,
is nonergodic and the mutual information between the blocks is bounded by the
logarithm of the block length. The second example is also nonergodic but the
mutual information between the blocks obeys a power law. The third example
obeys the power law and is ergodic.
