  Reinforcement learning (RL) and model predictive control (MPC) each offer
distinct advantages and limitations when applied to control problems in power
and energy systems. Despite various studies on these methods, benchmarks remain
lacking and the preference for RL over traditional controls is not well
understood. In this work, we put forth a comparative analysis using RL- and
MPC-based controllers for optimizing a battery management system (BMS). The BMS
problem aims to minimize costs while adhering to operational limits. by
adjusting the battery (dis)charging in response to fluctuating electricity
prices over a time horizon. The MPC controller uses a learningbased forecast of
future demand and price changes to formulate a multi-period linear program,
that can be solved using off-the-shelf solvers. Meanwhile, the RL controller
requires no timeseries modeling but instead is trained from the sample
trajectories using the proximal policy optimization (PPO) algorithm. Numerical
tests compare these controllers across optimality, training time, testing time,
and robustness, providing a comprehensive evaluation of their efficacy. RL not
only yields optimal solutions quickly but also ensures robustness to shifts in
customer behavior, such as changes in demand distribution. However, as
expected, training the RL agent is more time-consuming than MPC.
