  We consider the problem of learning a linear control policy for a linear
dynamical system, from demonstrations of an expert regulating the system. The
standard approach to this problem is policy fitting, which fits a linear policy
by minimizing a loss function between the demonstrations and the policy's
outputs plus a regularization function that encodes prior knowledge. Despite
its simplicity, this method fails to learn policies with low or even finite
cost when there are few demonstrations. We propose to add an additional
constraint to policy fitting, that the policy is the solution to some LQR
problem, i.e., optimal in the stochastic control sense for some choice of
quadratic cost. We refer to this constraint as a Kalman constraint. Policy
fitting with a Kalman constraint requires solving an optimization problem with
convex cost and bilinear constraints. We propose a heuristic method, based on
the alternating direction method of multipliers (ADMM), to approximately solve
this problem. Numerical experiments demonstrate that adding the Kalman
constraint allows us to learn good, i.e., low cost, policies even when very few
data are available.
