  This paper studies a deep deterministic policy gradient (DDPG) based actor
critic (AC) reinforcement learning (RL) technique to control a linear
discrete-time system with a quadratic control cost while ensuring a constraint
on the probability of potentially risky or undesirable events. The proposed
methodology can be applied to both known and unknown system models with minor
adjustments to the reward structure (negative cost). The problem is formulated
by considering the average expected quadratic cost of the states and inputs
over an infinite time horizon. Risky or undesirable events are represented as
functions of the states at the next time step exceeding a user-defined limit.
Two strategies are employed to manage the probabilistic constraint in scenarios
of known and unknown system models. In the case of a known system model, the
probabilistic constraint is replaced with an upper bound, such as the Chernoff
bound. For unknown system models, the expected value of the indicator function
of the occurrence of the risky or undesirable event is used. We have adopted a
deterministic policy gradient (DPG) based AC method to derive a parameterised
optimal policy. Extensive numerical simulations are performed using a second-
and a fourth-order system, and the proposed method is compared with the
standard risk-neutral linear quadratic regulator (LQR) and a chance-constrained
model predictive control (MPC) method. The results demonstrate the
effectiveness of the proposed approach in both known and unknown system model
scenarios.
