  Modern power systems will have to face difficult challenges in the years to
come: frequent blackouts in urban areas caused by high power demand peaks, grid
instability exacerbated by intermittent renewable generation, and global
climate change amplified by rising carbon emissions. While current practices
are growingly inadequate, the path to widespread adoption of artificial
intelligence (AI) methods is hindered by missing aspects of trustworthiness.
The CityLearn Challenge is an exemplary opportunity for researchers from
multiple disciplines to investigate the potential of AI to tackle these
pressing issues in the energy domain, collectively modeled as a reinforcement
learning (RL) task. Multiple real-world challenges faced by contemporary RL
techniques are embodied in the problem formulation. In this paper, we present a
novel method using the solution function of optimization as policies to compute
actions for sequential decision-making, while notably adapting the parameters
of the optimization model from online observations. Algorithmically, this is
achieved by an evolutionary algorithm under a novel trajectory-based guidance
scheme. Formally, the global convergence property is established. Our agent
ranked first in the latest 2021 CityLearn Challenge, being able to achieve
superior performance in almost all metrics while maintaining some key aspects
of interpretability.
