  Reinforcement Learning (RL) and continuous nonlinear control have been
successfully deployed in multiple domains of complicated sequential
decision-making tasks. However, given the exploration nature of the learning
process and the presence of model uncertainty, it is challenging to apply them
to safety-critical control tasks due to the lack of safety guarantee. On the
other hand, while combining control-theoretical approaches with learning
algorithms has shown promise in safe RL applications, the sample efficiency of
safe data collection process for control is not well addressed. In this paper,
we propose a \emph{provably} sample efficient episodic safe learning framework
for online control tasks that leverages safe exploration and exploitation in an
unknown, nonlinear dynamical system. In particular, the framework 1) extends
control barrier functions (CBFs) in a stochastic setting to achieve provable
high-probability safety under uncertainty during model learning and 2)
integrates an optimism-based exploration strategy to efficiently guide the safe
exploration process with learned dynamics for \emph{near optimal} control
performance. We provide formal analysis on the episodic regret bound against
the optimal controller and probabilistic safety with theoretical guarantees.
Simulation results are provided to demonstrate the effectiveness and efficiency
of the proposed algorithm.
