  Reinforcement learning has been successfully applied to the problem of tuning
PID controllers in several applications. The existing methods often utilize
function approximation, such as neural networks, to update the controller
parameters at each time-step of the underlying process. In this work, we
present a simple finite-difference approach, based on random search, to tuning
linear fixed-structure controllers. For clarity and simplicity, we focus on PID
controllers. Our algorithm operates on the entire closed-loop step response of
the system and iteratively improves the PID gains towards a desired closed-loop
response. This allows for embedding stability requirements into the reward
function without any modeling procedures.
