  This paper studies a model for online job scheduling in green datacenters. In
green datacenters, resource availability depends on the power supply from the
renewables. Intermittent power supply from renewables leads to intermittent
resource availability, inducing job delays (and associated costs). Green
datacenter operators must intelligently manage their workloads and available
power supply to extract maximum benefits. The scheduler's objective is to
schedule jobs on a set of resources to maximize the total value (revenue) while
minimizing the overall job delay. A trade-off exists between achieving high job
value on the one hand and low expected delays on the other. Hence, the aims of
achieving high rewards and low costs are in opposition. In addition, datacenter
operators often prioritize multiple objectives, including high system
utilization and job completion. To accomplish the opposing goals of maximizing
total job value and minimizing job delays, we apply the
Proportional-Integral-Derivative (PID) Lagrangian methods in Deep Reinforcement
Learning to job scheduling problem in the green datacenter environment.
Lagrangian methods are widely used algorithms for constrained optimization
problems. We adopt a controls perspective to learn the Lagrange multiplier with
proportional, integral, and derivative control, achieving favorable learning
dynamics. Feedback control defines cost terms for the learning agent, monitors
the cost limits during training, and continuously adjusts the learning
parameters to achieve stable performance. Our experiments demonstrate improved
performance compared to scheduling policies without the PID Lagrangian methods.
Experimental results illustrate the effectiveness of the Constraint Controlled
Reinforcement Learning (CoCoRL) scheduler that simultaneously satisfies
multiple objectives.
