  We consider policy gradient algorithms for the indefinite least squares
stationary optimal control, e.g., linear-quadratic-regulator (LQR) with
indefinite state and input penalization matrices. Such a setup has important
applications in control design with conflicting objectives, such as linear
quadratic dynamic games. We show the global convergence of gradient, natural
gradient and quasi-Newton policies for this class of indefinite least squares
problems.
