  This work examines the use of reinforcement learning (RL) to optimize cyclic
lockdowns, which is one of the methods available for control of the COVID-19
pandemic. The problem is structured as an optimal control system for tracking a
reference value, corresponding to the maximum usage level of a critical
resource, such as ICU beds. However, instead of using conventional optimal
control methods, RL is used to find optimal control policies. A framework was
developed to calculate optimal cyclic lockdown timings using an RL-based on-off
controller. The RL-based controller is implemented as an RL agent that
interacts with an epidemic simulator, implemented as an extended SEIR epidemic
model. The RL agent learns a policy function that produces an optimal sequence
of open/lockdown decisions such that goals specified in the RL reward function
are optimized. Two concurrent goals were used: the first one is a public health
goal that minimizes overshoots of ICU bed usage above an ICU bed threshold, and
the second one is a socio-economic goal that minimizes the time spent under
lockdowns. It is assumed that cyclic lockdowns are considered as a temporary
alternative to extended lockdowns when a region faces imminent danger of
overpassing resource capacity limits and when imposing an extended lockdown
would cause severe social and economic consequences due to lack of necessary
economic resources to support its affected population during an extended
lockdown.
