  Q-learning is a promising method for solving optimal control problems for
uncertain systems without the explicit need for system identification. However,
approaches for continuous-time Q-learning have limited provable safety
guarantees, which restrict their applicability to real-time safety-critical
systems. This paper proposes a safe Q-learning algorithm for partially unknown
linear time-invariant systems to solve the linear quadratic regulator problem
with user-defined state constraints. We frame the safe Q-learning problem as a
constrained optimal control problem using reciprocal control barrier functions
and show that such an extension provides a safety-assured control policy. To
the best of our knowledge, Q-learning for continuous-time systems with state
constraints has not yet been reported in the literature.
