  Replacing poorly performing existing controllers with smarter solutions will
decrease the energy intensity of the building sector. Recently, controllers
based on Deep Reinforcement Learning (DRL) have been shown to be more effective
than conventional baselines. However, since the optimal solution is usually
unknown, it is still unclear if DRL agents are attaining near-optimal
performance in general or if there is still a large gap to bridge.
  In this paper, we investigate the performance of DRL agents compared to the
theoretically optimal solution. To that end, we leverage Physically Consistent
Neural Networks (PCNNs) as simulation environments, for which optimal control
inputs are easy to compute. Furthermore, PCNNs solely rely on data to be
trained, avoiding the difficult physics-based modeling phase, while retaining
physical consistency. Our results hint that DRL agents not only clearly
outperform conventional rule-based controllers, they furthermore attain
near-optimal performance.
