  In this work, we propose a hierarchical reinforcement learning (HRL)
structure which is capable of performing autonomous vehicle planning tasks in
simulated environments with multiple sub-goals. In this hierarchical structure,
the network is capable of 1) learning one task with multiple sub-goals
simultaneously; 2) extracting attentions of states according to changing
sub-goals during the learning process; 3) reusing the well-trained network of
sub-goals for other similar tasks with the same sub-goals. The states are
defined as processed observations which are transmitted from the perception
system of the autonomous vehicle. A hybrid reward mechanism is designed for
different hierarchical layers in the proposed HRL structure. Compared to
traditional RL methods, our algorithm is more sample-efficient since its
modular design allows reusing the policies of sub-goals across similar tasks.
The results show that the proposed method converges to an optimal policy faster
than traditional RL methods.
