  This paper addresses the inverse optimal control problem of finding the state
weighting function that leads to a quadratic value function when the cost on
the input is fixed to be quadratic. The paper focuses on a class of infinite
horizon discrete-time and continuous-time optimal control problems whose
dynamics are control-affine and whose cost is quadratic in the input. The
optimal control policy for this problem is the projection of minus the gradient
of the value function onto the space formed by all feasible control directions.
This projection points along the control direction of steepest decrease of the
value function. For discrete-time systems and a quadratic value function the
optimal control law can be obtained as the solution of a regularized least
squares program, which corresponds to a receding horizon control with a single
step ahead. For the single input case and a quadratic value function the
solution for small weights in the control energy is interpreted as a control
policy that at each step brings the trajectories of the system as close as
possible to the origin, as measured by an appropriate norm. Conditions under
which the optimal control law is linear are also stated. Additionally, the
paper offers a mapping of the optimal control formulation to an equivalent
reinforcement learning formulation. Examples show the application of the
theoretical results.
