  We present an approach for agents to learn representations of a global map
from sensor data, to aid their exploration in new environments. To achieve
this, we embed procedures mimicking that of traditional Simultaneous
Localization and Mapping (SLAM) into the soft attention based addressing of
external memory architectures, in which the external memory acts as an internal
representation of the environment. This structure encourages the evolution of
SLAM-like behaviors inside a completely differentiable deep neural network. We
show that this approach can help reinforcement learning agents to successfully
explore new environments where long-term memory is essential. We validate our
approach in both challenging grid-world environments and preliminary Gazebo
experiments. A video of our experiments can be found at: https://goo.gl/G2Vu5y.
