  Deep reinforcement learning (RL) has brought many successes for autonomous
robot navigation. However, there still exists important limitations that
prevent real-world use of RL-based navigation systems. For example, most
learning approaches lack safety guarantees; and learned navigation systems may
not generalize well to unseen environments. Despite a variety of recent
learning techniques to tackle these challenges in general, a lack of an
open-source benchmark and reproducible learning methods specifically for
autonomous navigation makes it difficult for roboticists to choose what
learning methods to use for their mobile robots and for learning researchers to
identify current shortcomings of general learning methods for autonomous
navigation. In this paper, we identify four major desiderata of applying deep
RL approaches for autonomous navigation: (D1) reasoning under uncertainty, (D2)
safety, (D3) learning from limited trial-and-error data, and (D4)
generalization to diverse and novel environments. Then, we explore four major
classes of learning techniques with the purpose of achieving one or more of the
four desiderata: memory-based neural network architectures (D1), safe RL (D2),
model-based RL (D2, D3), and domain randomization (D4). By deploying these
learning techniques in a new open-source large-scale navigation benchmark and
real-world environments, we perform a comprehensive study aimed at establishing
to what extent can these techniques achieve these desiderata for RL-based
navigation systems.
