Multi-Agent Deep Reinforcement Learning for HVAC Control in Commercial Buildings

by   Liang Yu, et al.

In commercial buildings, about 40 is attributed to Heating, Ventilation, and Air Conditioning (HVAC) systems, which places an economic burden on building operators. In this paper, we intend to minimize the energy cost of an HVAC system in a multi-zone commercial building under dynamic pricing with the consideration of random zone occupancy, thermal comfort, and indoor air quality comfort. Due to the existence of unknown thermal dynamics models, parameter uncertainties (e.g., outdoor temperature, electricity price, and number of occupants), spatially and temporally coupled constraints associated with indoor temperature and CO2 concentration, a large discrete solution space, and a non-convex and non-separable objective function, it is very challenging to achieve the above aim. To this end, the above energy cost minimization problem is reformulated as a Markov game. Then, an HVAC control algorithm is proposed to solve the Markov game based on multi-agent deep reinforcement learning with attention mechanism. The proposed algorithm does not require any prior knowledge of uncertain parameters and can operate without knowing building thermal dynamics models. Simulation results based on real-world traces show the effectiveness, robustness and scalability of the proposed algorithm.


page 1

page 2

page 3

page 5

page 6

page 9

page 11

page 13


Distributed Multi-Agent Deep Reinforcement Learning Framework for Whole-building HVAC Control

It is estimated that about 40 commercial buildings can be attributed to ...

Multi-agent Reinforcement Learning Embedded Game for the Optimization of Building Energy Control and Power System Planning

Most of the current game-theoretic demand-side management methods focus ...

End-to-end deep metamodeling to calibrate and optimize energy loads

In this paper, we propose a new end-to-end methodology to optimize the e...

Statistical Mechanics of Thermostatically Controlled Multi-Zone Buildings

We study the collective phenomena and constraints associated with the ag...

Deep Reinforcement Learning for Optimal Control of Space Heating

Classical methods to control heating systems are often marred by subopti...

Estimating Buildings' Parameters over Time Including Prior Knowledge

Modeling buildings' heat dynamics is a complex process which depends on ...

Window Opening Model using Deep Learning Methods

Occupant behavior (OB) and in particular window openings need to be cons...

Please sign up or login with your details

Forgot password? Click here to reset