  Unobservable Markov decision processes (UMDPs) serve as a prominent
mathematical framework for modeling sequential decision-making problems. A key
aspect in computational analysis is the consideration of decidability, which
concerns the existence of algorithms. In general, the computation of the exact
and approximated values is undecidable for UMDPs with the long-run average
objective. Building on matrix product theory and ergodic properties, we
introduce a novel subclass of UMDPs, termed ergodic UMDPs. Our main result
demonstrates that approximating the value within this subclass is decidable.
However, we show that the exact problem remains undecidable. Finally, we
discuss the primary challenges of extending these results to partially
observable Markov decision processes.
