  Reinforcement Learning faces an important challenge in partial observable
environments that has long-term dependencies. In order to learn in an ambiguous
environment, an agent has to keep previous perceptions in a memory. Earlier
memory based approaches use a fixed method to determine what to keep in the
memory, which limits them to certain problems. In this study, we follow the
idea of giving the control of the memory to the agent by allowing it to have
memory-changing actions. This learning mechanism is supported by an intrinsic
motivation to memorize rare observations that can help the agent to
disambiguate its state in the environment. Our approach is experimented and
analyzed on several partial observable tasks with long-term dependencies and
compared with other memory based methods.
