  Reinforcement learning (RL) algorithms can achieve state-of-the-art
performance in decision-making and continuous control tasks. However, applying
RL algorithms on safety-critical systems still needs to be well justified due
to the exploration nature of many RL algorithms, especially when the model of
the robot and the environment are unknown. To address this challenge, we
propose a data-driven safety layer that acts as a filter for unsafe actions.
The safety layer uses a data-driven predictive controller to enforce safety
guarantees for RL policies during training and after deployment. The RL agent
proposes an action that is verified by computing the data-driven reachability
analysis. If there is an intersection between the reachable set of the robot
using the proposed action, we call the data-driven predictive controller to
find the closest safe action to the proposed unsafe action. The safety layer
penalizes the RL agent if the proposed action is unsafe and replaces it with
the closest safe one. In the simulation, we show that our method outperforms
state-of-the-art safe RL methods on the robotics navigation problem for a
Turtlebot 3 in Gazebo and a quadrotor in Unreal Engine 4 (UE4).
