  Planning safe trajectories under uncertain and dynamic conditions makes the
autonomous driving problem significantly complex. Current sampling-based
methods such as Rapidly Exploring Random Trees (RRTs) are not ideal for this
problem because of the high computational cost. Supervised learning methods
such as Imitation Learning lack generalization and safety guarantees. To
address these problems and in order to ensure a robust framework, we propose a
Hierarchical Reinforcement Learning (HRL) structure combined with a
Proportional-Integral-Derivative (PID) controller for trajectory planning. HRL
helps divide the task of autonomous vehicle driving into sub-goals and supports
the network to learn policies for both high-level options and low-level
trajectory planner choices. The introduction of sub-goals decreases convergence
time and enables the policies learned to be reused for other scenarios. In
addition, the proposed planner is made robust by guaranteeing smooth
trajectories and by handling the noisy perception system of the ego-car. The
PID controller is used for tracking the waypoints, which ensures smooth
trajectories and reduces jerk. The problem of incomplete observations is
handled by using a Long-Short-Term-Memory (LSTM) layer in the network. Results
from the high-fidelity CARLA simulator indicate that the proposed method
reduces convergence time, generates smoother trajectories, and is able to
handle dynamic surroundings and noisy observations.
