  Model-Based Reinforcement Learning (MBRL) has been widely studied for
Heating, Ventilation, and Air Conditioning (HVAC) control in buildings. One of
the critical challenges is the large amount of data required to effectively
train neural networks for modeling building dynamics. This paper presents CLUE,
an MBRL system for HVAC control in buildings. CLUE optimizes HVAC operations by
integrating a Gaussian Process (GP) model to model building dynamics with
uncertainty awareness. CLUE utilizes GP to predict state transitions as
Gaussian distributions, effectively capturing prediction uncertainty and
enhancing decision-making under sparse data conditions. Our approach employs a
meta-kernel learning technique to efficiently set GP kernel hyperparameters
using domain knowledge from diverse buildings. This drastically reduces the
data requirements typically associated with GP models in HVAC applications.
Additionally, CLUE incorporates these uncertainty estimates into a Model
Predictive Path Integral (MPPI) algorithm, enabling the selection of safe,
energy-efficient control actions. This uncertainty-aware control strategy
evaluates and selects action trajectories based on their predicted impact on
energy consumption and human comfort, optimizing operations even under
uncertain conditions. Extensive simulations in a five-zone office building
demonstrate that CLUE reduces the required training data from hundreds of days
to just seven while maintaining robust control performance. It reduces comfort
violations by an average of 12.07% compared to existing MBRL methods, without
compromising on energy efficiency.
