  Deep reinforcement learning (DRL) has a great potential for solving complex
decision-making problems in autonomous driving, especially in mixed-traffic
scenarios where autonomous vehicles and human-driven vehicles (HDVs) drive
together. Safety is a key during both the learning and deploying reinforcement
learning (RL) algorithms process. In this paper, we formulate the on-ramp
merging as a Markov Decision Process (MDP) problem and solve it with an
off-policy RL algorithm, i.e., Soft Actor-Critic for Discrete Action Settings
(SAC-Discrete). In addition, a motion predictive safety controller including a
motion predictor and an action substitution module, is proposed to ensure
driving safety during both training and testing. The motion predictor estimates
the trajectories of the ego vehicle and surrounding vehicles from kinematic
models, and predicts potential collisions. The action substitution module
updates the actions based on safety distance and replaces risky actions, before
sending them to the low-level controller. We train, evaluate and test our
approach on a gym-like highway simulation with three different levels of
traffic modes. The simulation results show that even in harder traffic
densities, our proposed method still significantly reduces collision rate while
maintaining high efficiency, outperforming several state-of-the-art baselines
in the considered on-ramp merging scenarios. The video demo of the evaluation
process can be found at: https://www.youtube.com/watch?v=7FvjbAM4oFw
