  It is estimated that about 40%-50% of total electricity consumption in
commercial buildings can be attributed to Heating, Ventilation, and Air
Conditioning (HVAC) systems. Minimizing the energy cost while considering the
thermal comfort of the occupants is very challenging due to unknown and complex
relationships between various HVAC controls and thermal dynamics inside a
building. To this end, we present a multi-agent, distributed deep reinforcement
learning (DRL) framework based on Energy Plus simulation environment for
optimizing HVAC in commercial buildings. This framework learns the complex
thermal dynamics in the building and takes advantage of the differential effect
of cooling and heating systems in the building to reduce energy costs, while
maintaining the thermal comfort of the occupants. With adaptive penalty, the RL
algorithm can be prioritized for energy savings or maintaining thermal comfort.
Using DRL, we achieve more than 75\% savings in energy consumption. The
distributed DRL framework can be scaled to multiple GPUs and CPUs of
heterogeneous types.
