  Large-scale integration of intermittent renewable energy sources calls for
substantial demand side flexibility. Given that the built environment accounts
for approximately 40% of total energy consumption in EU, unlocking its
flexibility is a key step in the energy transition process. This paper focuses
specifically on energy flexibility in residential buildings, leveraging their
intrinsic thermal mass. Building on recent developments in the field of
data-driven control, we propose PhysQ. As a physics-informed reinforcement
learning framework for building control, PhysQ forms a step in bridging the gap
between conventional model-based control and data-intensive control based on
reinforcement learning. Through our experiments, we show that the proposed
PhysQ framework can learn high quality control policies that outperform a
business-as-usual, as well as a rudimentary model predictive controller. Our
experiments indicate cost savings of about 9% compared to a business-as-usual
controller. Further, we show that PhysQ efficiently leverages prior physics
knowledge to learn such policies using fewer training samples than conventional
reinforcement learning approaches, making PhysQ a scalable alternative for use
in residential buildings. Additionally, the PhysQ control policy utilizes
building state representations that are intuitive and based on conventional
building models, that leads to better interpretation of the learnt policy over
other data-driven controllers.
