  To reduce global carbon emissions and limit climate change, controlling
energy consumption in buildings is an important piece of the puzzle. Here, we
specifically focus on using a demand response (DR) algorithm to limit the
energy consumption of a residential building's heating system while respecting
user's thermal comfort. In that domain, Reinforcement learning (RL) methods
have been shown to be quite effective. One such RL method is Monte Carlo Tree
Search (MCTS), which has achieved impressive success in playing board games
(go, chess). A particular advantage of MCTS is that its decision tree structure
naturally allows to integrate exogenous constraints (e.g., by trimming branches
that violate them), while conventional RL solutions need more elaborate
techniques (e.g., indirectly by adding penalties in the cost/reward function,
or through a backup controller that corrects constraint-violating actions). The
main aim of this paper is to study the adoption of MCTS for building control,
since this (to the best of our knowledge) has remained largely unexplored. A
specific property of MCTS is that it needs a simulator component that can
predict subsequent system states, based on actions taken. A straightforward
data-driven solution is to use black-box neural networks (NNs). We will however
extend a Physics-informed Neural Network (PiNN) model to deliver multi-timestep
predictions, and show the benefit it offers in terms of lower prediction errors
($-$32\% MAE) as well as better MCTS performance ($-$4\% energy cost, $+$7\%
thermal comfort) compared to a black-box NN. A second contribution will be to
extend a vanilla MCTS version to adopt the ideas applied in AlphaZero (i.e.,
using learned prior and value functions and an action selection heuristic) to
obtain lower computational costs while maintaining control performance.
