  In this paper, we investigate an energy cost minimization problem for a smart
home in the absence of a building thermal dynamics model with the consideration
of a comfortable temperature range. Due to the existence of model uncertainty,
parameter uncertainty (e.g., renewable generation output, non-shiftable power
demand, outdoor temperature, and electricity price) and temporally-coupled
operational constraints, it is very challenging to determine the optimal energy
management strategy for scheduling Heating, Ventilation, and Air Conditioning
(HVAC) systems and energy storage systems in the smart home. To address the
challenge, we first formulate the above problem as a Markov decision process,
and then propose an energy management strategy based on Deep Deterministic
Policy Gradients (DDPG). It is worth mentioning that the proposed strategy does
not require the prior knowledge of uncertain parameters and building thermal
dynamics model. Simulation results based on real-world traces demonstrate the
effectiveness and robustness of the proposed strategy.
