  Reinforcement learning is commonly applied in residential energy management,
particularly for optimizing energy costs. However, RL agents often face
challenges when dealing with deceptive and sparse rewards in the energy control
domain, especially with stochastic rewards. In such situations, thorough
exploration becomes crucial for learning an optimal policy. Unfortunately, the
exploration mechanism can be misled by deceptive reward signals, making
thorough exploration difficult. Go-Explore is a family of algorithms which
combines planning methods and reinforcement learning methods to achieve
efficient exploration. We use the Go-Explore algorithm to solve the cost-saving
task in residential energy management problems and achieve an improvement of up
to 19.84\% compared to the well-known reinforcement learning algorithms.
