  Learning algorithms have shown considerable prowess in simulation by allowing
robots to adapt to uncertain environments and improve their performance.
However, such algorithms are rarely used in practice on safety-critical
systems, since the learned policy typically does not yield any safety
guarantees. That is, the required exploration may cause physical harm to the
robot or its environment. In this paper, we present a method to learn accurate
safety certificates for nonlinear, closed-loop dynamical systems. Specifically,
we construct a neural network Lyapunov function and a training algorithm that
adapts it to the shape of the largest safe region in the state space. The
algorithm relies only on knowledge of inputs and outputs of the dynamics,
rather than on any specific model structure. We demonstrate our method by
learning the safe region of attraction for a simulated inverted pendulum.
Furthermore, we discuss how our method can be used in safe learning algorithms
together with statistical models of dynamical systems.
