  This paper presents a novel approach that supports natural language voice
instructions to guide deep reinforcement learning (DRL) algorithms when
training self-driving cars. DRL methods are popular approaches for autonomous
vehicle (AV) agents. However, most existing methods are sample- and
time-inefficient and lack a natural communication channel with the human
expert. In this paper, how new human drivers learn from human coaches motivates
us to study new ways of human-in-the-loop learning and a more natural and
approachable training interface for the agents. We propose incorporating
natural language voice instructions (NLI) in model-based deep reinforcement
learning to train self-driving cars. We evaluate the proposed method together
with a few state-of-the-art DRL methods in the CARLA simulator. The results
show that NLI can help ease the training process and significantly boost the
agents' learning speed.
