  This paper develops a model-based reinforcement learning (MBRL) framework for
learning online the value function of an infinite-horizon optimal control
problem while obeying safety constraints expressed as control barrier functions
(CBFs). Our approach is facilitated by the development of a novel class of
CBFs, termed Lyapunov-like CBFs (LCBFs), that retain the beneficial properties
of CBFs for developing minimally-invasive safe control policies while also
possessing desirable Lyapunov-like qualities such as positive
semi-definiteness. We show how these LCBFs can be used to augment a
learning-based control policy to guarantee safety and then leverage this
approach to develop a safe exploration framework in a MBRL setting. We
demonstrate that our approach can handle more general safety constraints than
comparative methods via numerical examples.
