A learning agent for heat-pump thermostat control

Heating, Ventilation and Air Conditioning (HVAC) systems are one of the biggest energy consumers around the world. With the efforts of moving to sustainable energy consumption, heat-pump based HVAC systems have gained popularity due to their high efficiency and due to the fact that they are powered by electricity rather than by gas or oil. One drawback of heat-pump systems is that their efficiency sharply decreases when the outdoor temperature is around or below freezing. Therefore, they are backed up by an auxiliary heating system that is effective in cold weather but that consumes twice as much energy. A popular way of saving energy in HVAC systems is setting back the thermostat, meaning, relaxing the heating/cooling requirements when occupants are not at home. While this practice leads to significant energy savings in many systems, it could in fact increase the energy consumption in a heat-pump based system, using existing control strategies, as it forces an excessive usage of the auxiliary heater. In this paper, we design and implement a complete, adaptive reinforcement learning agent which applies a new control strategy for a heat-pump thermostat. For our experiments, we use a complex, realistic simulator that was developed for the US Department of Energy. Results show that the learned control strategy (1) leads to roughly 7.0%-14.5% energy savings in typical homes in the New York City, Boston, and Chicago areas; while (2) keeping the occupants’ comfort level unchanged when compared to an existing strategy that is deployed in practice.