  This paper demonstrates that continual relearning of control policies using
incremental deep reinforcement learning (RL) can improve policy learning for
non-stationary processes. We demonstrate this approach for a data-driven 'smart
building environment' that we use as a test-bed for developing HVAC controllers
for reducing energy consumption of large buildings on our university campus.
The non-stationarity in building operations and weather patterns makes it
imperative to develop control strategies that are adaptive to changing
conditions. On-policy RL algorithms, such as Proximal Policy Optimization (PPO)
represent an approach for addressing this non-stationarity, but exploration on
the actual system is not an option for safety-critical systems. As an
alternative, we develop an incremental RL technique that simultaneously reduces
building energy consumption without sacrificing overall comfort. We compare the
performance of our incremental RL controller to that of a static RL controller
that does not implement the relearning function. The performance of the static
controller diminishes significantly over time, but the relearning controller
adjusts to changing conditions while ensuring comfort and optimal energy
performance.
