  Reinforcement learning (RL) methods for social robot navigation show great
success navigating robots through large crowds of people, but the performance
of these learning-based methods tends to degrade in particularly challenging or
unfamiliar situations due to the models' dependency on representative training
data. To ensure human safety and comfort, it is critical that these algorithms
handle uncommon cases appropriately, but the low frequency and wide diversity
of such situations present a significant challenge for these data-driven
methods. To overcome this challenge, we propose modifications to the learning
process that encourage these RL policies to maintain additional caution in
unfamiliar situations. Specifically, we improve the Socially Attentive
Reinforcement Learning (SARL) policy by (1) modifying the training process to
systematically introduce deviations into a pedestrian model, (2) updating the
value network to estimate and utilize pedestrian-unpredictability features, and
(3) implementing a reward function to learn an effective response to pedestrian
unpredictability. Compared to the original SARL policy, our modified policy
maintains similar navigation times and path lengths, while reducing the number
of collisions by 82% and reducing the proportion of time spent in the
pedestrians' personal space by up to 19 percentage points for the most
difficult cases. We also describe how to apply these modifications to other RL
policies and demonstrate that some key high-level behaviors of our approach
transfer to a physical robot.
