  Autonomous driving is an emerging technology that has advanced rapidly over
the last decade. Modern transportation is expected to benefit greatly from a
wise decision-making framework of autonomous vehicles, including the
improvement of mobility and the minimization of risks and travel time. However,
existing methods either ignore the complexity of environments only fitting
straight roads, or ignore the impact on surrounding vehicles during
optimization phases, leading to weak environmental adaptability and incomplete
optimization objectives. To address these limitations, we propose a
parameterized decision-making framework with multi-modal perception based on
deep reinforcement learning, called AUTO. We conduct a comprehensive perception
to capture the state features of various traffic participants around the
autonomous vehicle, based on which we design a graph-based model to learn a
state representation of the multi-modal semantic features. To distinguish
between lane-following and lane-changing, we decompose an action of the
autonomous vehicle into a parameterized action structure that first decides
whether to change lanes and then computes an exact action to execute. A hybrid
reward function takes into account aspects of safety, traffic efficiency,
passenger comfort, and impact to guide the framework to generate optimal
actions. In addition, we design a regularization term and a multi-worker
paradigm to enhance the training. Extensive experiments offer evidence that
AUTO can advance state-of-the-art in terms of both macroscopic and microscopic
effectiveness.
