  Vanilla Reinforcement Learning (RL) can efficiently solve complex tasks but
does not provide any guarantees on system behavior. To bridge this gap, we
propose a three-step safe RL procedure for continuous action spaces that
provides probabilistic guarantees with respect to temporal logic
specifications. First, our approach probabilistically verifies a candidate
controller with respect to a temporal logic specification while randomizing the
control inputs to the system within a bounded set. Second, we improve the
performance of this probabilistically verified controller by adding an RL agent
that optimizes the verified controller for performance in the same bounded set
around the control input. Third, we verify probabilistic safety guarantees with
respect to temporal logic specifications for the learned agent. Our approach is
efficiently implementable for continuous action and state spaces. The
separation of safety verification and performance improvement into two distinct
steps realizes both explicit probabilistic safety guarantees and a
straightforward RL setup that focuses on performance. We evaluate our approach
on an evasion task where a robot has to reach a goal while evading a dynamic
obstacle with a specific maneuver. Our results show that our safe RL approach
leads to efficient learning while maintaining its probabilistic safety
specification.
