  Driving in a dynamic, multi-agent, and complex urban environment is a
difficult task requiring a complex decision-making policy. The learning of such
a policy requires a state representation that can encode the entire
environment. Mid-level representations that encode a vehicle's environment as
images have become a popular choice. Still, they are quite high-dimensional,
limiting their use in data-hungry approaches such as reinforcement learning. In
this article, we propose to learn a low-dimensional and rich latent
representation of the environment by leveraging the knowledge of relevant
semantic factors. To do this, we train an encoder-decoder deep neural network
to predict multiple application-relevant factors such as the trajectories of
other agents and the ego car. Furthermore, we propose a hazard signal based on
other vehicles' future trajectories and the planned route which is used in
conjunction with the learned latent representation as input to a down-stream
policy. We demonstrate that using the multi-head encoder-decoder neural network
results in a more informative representation than a standard single-head model.
In particular, the proposed representation learning and the hazard signal help
reinforcement learning to learn faster, with increased performance and less
data than baseline methods.
