  Heating, Ventilation, and Air Conditioning (HVAC) systems are a major driver
of energy consumption in commercial and residential buildings. Recent studies
have shown that Deep Reinforcement Learning (DRL) algorithms can outperform
traditional reactive controllers. However, DRL-based solutions are generally
designed for ad hoc setups and lack standardization for comparison. To fill
this gap, this paper provides a critical and reproducible evaluation, in terms
of comfort and energy consumption, of several state-of-the-art DRL algorithms
for HVAC control. The study examines the controllers' robustness, adaptability,
and trade-off between optimization goals by using the Sinergym framework. The
results obtained confirm the potential of DRL algorithms, such as SAC and TD3,
in complex scenarios and reveal several challenges related to generalization
and incremental learning.
