  In this paper, an off-policy reinforcement learning algorithm is designed to
solve the continuous-time LQR problem using only input-state data measured from
the system. Different from other algorithms in the literature, we propose the
use of a specific persistently exciting input as the exploration signal during
the data collection step. We then show that, using this persistently excited
data, the solution of the matrix equation in our algorithm is guaranteed to
exist and to be unique at every iteration. Convergence of the algorithm to the
optimal control input is also proven. Moreover, we formulate the policy
evaluation step as the solution of a Sylvester-transpose equation, which
increases the efficiency of its solution. Finally, a method to determine a
stabilizing policy to initialize the algorithm using only measured data is
proposed.
