  Lane change is a challenging task which requires delicate actions to ensure
safety and comfort. Some recent studies have attempted to solve the lane-change
control problem with Reinforcement Learning (RL), yet the action is confined to
discrete action space. To overcome this limitation, we formulate the lane
change behavior with continuous action in a model-free dynamic driving
environment based on Deep Deterministic Policy Gradient (DDPG). The reward
function, which is critical for learning the optimal policy, is defined by
control values, position deviation status, and maneuvering time to provide the
RL agent informative signals. The RL agent is trained from scratch without
resorting to any prior knowledge of the environment and vehicle dynamics since
they are not easy to obtain. Seven models under different hyperparameter
settings are compared. A video showing the learning progress of the driving
behavior is available. It demonstrates the RL vehicle agent initially runs out
of road boundary frequently, but eventually has managed to smoothly and stably
change to the target lane with a success rate of 100% under diverse driving
situations in simulation.
