  Partially observable Markov decision processes (POMDPs) have been widely
applied in various real-world applications. However, existing theoretical
results have shown that learning in POMDPs is intractable in the worst case,
where the main challenge lies in the lack of latent state information. A key
fundamental question here is: how much online state information (OSI) is
sufficient to achieve tractability? In this paper, we establish a lower bound
that reveals a surprising hardness result: unless we have full OSI, we need an
exponentially scaling sample complexity to obtain an $\epsilon$-optimal policy
solution for POMDPs. Nonetheless, inspired by the insights in our lower-bound
design, we identify important tractable subclasses of POMDPs, even with only
partial OSI. In particular, for two subclasses of POMDPs with partial OSI, we
provide new algorithms that are proved to be near-optimal by establishing new
regret upper and lower bounds. Both our algorithm design and regret analysis
involve non-trivial developments for joint OSI query and action control.
