  In real-world reinforcement learning (RL) systems, various forms of {\it
impaired observability} can complicate matters. These situations arise when an
agent is unable to observe the most recent state of the system due to latency
or lossy channels, yet the agent must still make real-time decisions. This
paper introduces a theoretical investigation into efficient RL in control
systems where agents must act with delayed and missing state observations. We
present algorithms and establish near-optimal regret upper and lower bounds, of
the form $\tilde{\mathcal{O}}(\sqrt{{\rm poly}(H) SAK})$, for RL in the delayed
and missing observation settings. Here $S$ and $A$ are the sizes of state and
action spaces, $H$ is the time horizon and $K$ is the number of episodes.
Despite impaired observability posing significant challenges to the policy
class and planning, our results demonstrate that learning remains efficient,
with the regret bound optimally depending on the state-action size of the
original system. Additionally, we provide a characterization of the performance
of the optimal policy under impaired observability, comparing it to the optimal
value obtained with full observability. Numerical results are provided to
support our theory.
