  For safely applying reinforcement learning algorithms on high-dimensional
nonlinear dynamical systems, a simplified system model is used to formulate a
safe reinforcement learning framework. Based on the simplified system model, a
low-dimensional representation of the safe region is identified and is used to
provide safety estimates for learning algorithms. However, finding a satisfying
simplified system model for complex dynamical systems usually requires a
considerable amount of effort. To overcome this limitation, we propose in this
work a general data-driven approach that is able to efficiently learn a
low-dimensional representation of the safe region. Through an online adaptation
method, the low-dimensional representation is updated by using the feedback
data such that more accurate safety estimates are obtained. The performance of
the proposed approach for identifying the low-dimensional representation of the
safe region is demonstrated with a quadcopter example. The results show that,
compared to previous work, a more reliable and representative low-dimensional
representation of the safe region is derived, which then extends the
applicability of the safe reinforcement learning framework.
