  Overtaking on two-lane roads is a great challenge for autonomous vehicles, as
oncoming traffic appearing on the opposite lane may require the vehicle to
change its decision and abort the overtaking. Deep reinforcement learning (DRL)
has shown promise for difficult decision problems such as this, but it requires
massive number of data, especially if the action space is continuous. This
paper proposes to incorporate guidance from an expert system into DRL to
increase its sample efficiency in the autonomous overtaking setting. The
guidance system developed in this study is composed of constrained iterative
LQR and PID controllers. The novelty lies in the incorporation of a fading
guidance function, which gradually decreases the effect of the expert system,
allowing the agent to initially learn an appropriate action swiftly and then
improve beyond the performance of the expert system. This approach thus
combines the strengths of traditional control engineering with the flexibility
of learning systems, expanding the capabilities of the autonomous system. The
proposed methodology for autonomous vehicle overtaking does not depend on a
particular DRL algorithm and three state-of-the-art algorithms are used as
baselines for evaluation. Simulation results show that incorporating expert
system guidance improves state-of-the-art DRL algorithms greatly in both sample
efficiency and driving safety.
