  This paper presented a deep reinforcement learning method named Double Deep
Q-networks to design an end-to-end vision-based adaptive cruise control (ACC)
system. A simulation environment of a highway scene was set up in Unity, which
is a game engine that provided both physical models of vehicles and feature
data for training and testing. Well-designed reward functions associated with
the following distance and throttle/brake force were implemented in the
reinforcement learning model for both internal combustion engine (ICE) vehicles
and electric vehicles (EV) to perform adaptive cruise control. The gap
statistics and total energy consumption are evaluated for different vehicle
types to explore the relationship between reward functions and powertrain
characteristics. Compared with the traditional radar-based ACC systems or
human-in-the-loop simulation, the proposed vision-based ACC system can generate
either a better gap regulated trajectory or a smoother speed trajectory
depending on the preset reward function. The proposed system can be well
adaptive to different speed trajectories of the preceding vehicle and operated
in real-time.
