  Experimentally comparing the energy usage and comfort characteristics of
different controllers in heating, ventilation, and air-conditioning (HVAC)
systems is difficult because variations in weather and occupancy conditions
preclude the possibility of establishing equivalent experimental conditions
across the order of hours, days, and weeks. This paper is concerned with
defining quantitative metrics of energy usage and occupant comfort, which can
be computed and compared in a rigorous manner that is capable of determining
whether differences between controllers are statistically significant in the
presence of such environmental fluctuations. Experimental case studies are
presented that compare two alternative controllers (a schedule controller and a
hybrid system learning-based model predictive controller) to the default
controller in a building-wide HVAC system. Lastly, we discuss how our proposed
methodology may also be able to quantify the efficiency of other building
automation systems.
