Addressing Inherent Uncertainty: Risk-Sensitive Behavior Generation for
  Automated Driving using Distributional Reinforcement Learning
  For highly automated driving above SAE level~3, behavior generation
algorithms must reliably consider the inherent uncertainties of the traffic
environment, e.g. arising from the variety of human driving styles. Such
uncertainties can generate ambiguous decisions, requiring the algorithm to
appropriately balance low-probability hazardous events, e.g. collisions, and
high-probability beneficial events, e.g. quickly crossing the intersection.
State-of-the-art behavior generation algorithms lack a distributional treatment
of decision outcome. This impedes a proper risk evaluation in ambiguous
situations, often encouraging either unsafe or conservative behavior. Thus, we
propose a two-step approach for risk-sensitive behavior generation combining
offline distribution learning with online risk assessment. Specifically, we
first learn an optimal policy in an uncertain environment with Deep
Distributional Reinforcement Learning. During execution, the optimal
risk-sensitive action is selected by applying established risk criteria, such
as the Conditional Value at Risk, to the learned state-action return
distributions. In intersection crossing scenarios, we evaluate different risk
criteria and demonstrate that our approach increases safety, while maintaining
an active driving style. Our approach shall encourage further studies about the
benefits of risk-sensitive approaches for self-driving vehicles.
