  The potential of Machine Learning Control (MLC) in HVAC systems is hindered
by its opaque nature and inference mechanisms, which is challenging for users
and modelers to fully comprehend, ultimately leading to a lack of trust in
MLC-based decision-making. To address this challenge, this paper investigates
and explores Interpretable Machine Learning (IML), a branch of Machine Learning
(ML) that enhances transparency and understanding of models and their
inferences, to improve the credibility of MLC and its industrial application in
HVAC systems. Specifically, we developed an innovative framework that combines
the principles of Shapley values and the in-context learning feature of Large
Language Models (LLMs). While the Shapley values are instrumental in dissecting
the contributions of various features in ML models, LLM provides an in-depth
understanding of rule-based parts in MLC; combining them, LLM further packages
these insights into a coherent, human-understandable narrative. The paper
presents a case study to demonstrate the feasibility of the developed IML
framework for model predictive control-based precooling under demand response
events in a virtual testbed. The results indicate that the developed framework
generates and explains the control signals in accordance with the rule-based
rationale.
