  Reinforcement learning (RL)-based adaptive cruise control systems (ACC) that
learn and adapt to road, traffic and vehicle conditions are attractive for
enhancing vehicle energy efficiency and traffic flow. However, the application
of RL in safety critical systems such as ACC requires strong safety guarantees
which are difficult to achieve with learning agents that have a fundamental
need to explore. In this paper, we derive control barrier functions as safety
filters that allow an RL-based ACC controller to explore freely within a
collision safe set. Specifically, we derive control barrier functions for high
relative degree nonlinear systems to take into account inertia effects relevant
to commercial vehicles. We also outline an algorithm for accommodating
actuation saturation with these barrier functions. While any RL algorithm can
be used as the performance ACC controller together with these filters, we
implement the Maximum A Posteriori Policy Optimization (MPO) algorithm with a
hybrid action space that learns fuel optimal gear selection and torque control
policies. The safety filtering RL approach is contrasted with a reward shaping
RL approach that only learns to avoid collisions after sufficient training.
Evaluations on different drive cycles demonstrate significant improvements in
fuel economy with the proposed approach compared to baseline ACC algorithms.
