  Reinforcement learning (RL) has been shown to learn sophisticated control
policies for complex tasks including games, robotics, heating and cooling
systems and text generation. The action-perception cycle in RL, however,
generally assumes that a measurement of the state of the environment is
available at each time step without a cost. In applications such as materials
design, deep-sea and planetary robot exploration and medicine, however, there
can be a high cost associated with measuring, or even approximating, the state
of the environment. In this paper, we survey the recently growing literature
that adopts the perspective that an RL agent might not need, or even want, a
costly measurement at each time step. Within this context, we propose the Deep
Dynamic Multi-Step Observationless Agent (DMSOA), contrast it with the
literature and empirically evaluate it on OpenAI gym and Atari Pong
environments. Our results, show that DMSOA learns a better policy with fewer
decision steps and measurements than the considered alternative from the
literature.
