  We investigate how a neural network can learn perception actions loops for
navigation in unknown environments. Specifically, we consider how to learn to
navigate in environments populated with cul-de-sacs that represent convex local
minima that the robot could fall into instead of finding a set of feasible
actions that take it to the goal. Traditional methods rely on maintaining a
global map to solve the problem of over coming a long cul-de-sac. However, due
to errors induced from local and global drift, it is highly challenging to
maintain such a map for long periods of time. One way to mitigate this problem
is by using learning techniques that do not rely on hand engineered map
representations and instead output appropriate control policies directly from
their sensory input. We first demonstrate that such a problem cannot be solved
directly by deep reinforcement learning due to the sparse reward structure of
the environment. Further, we demonstrate that deep supervised learning also
cannot be used directly to solve this problem. We then investigate network
models that offer a combination of reinforcement learning and supervised
learning and highlight the significance of adding fully differentiable memory
units to such networks. We evaluate our networks on their ability to generalize
to new environments and show that adding memory to such networks offers huge
jumps in performance
