Online Multi-agent Reinforcement Learning for Decentralized Inverter-based Volt-VAR Control

06/23/2020
by   Haotian Liu, et al.
0

The distributed Volt/Var control (VVC) methods have been widely studied for active distribution networks(ADNs), which is based on perfect model and real-time P2P communication. However, the model is always incomplete with significant parameter errors and such P2P communication system is hard to maintain. In this paper, we propose an online multi-agent reinforcement learning and decentralized control framework (OLDC) for VVC. In this framework, the VVC problem is formulated as a constrained Markov game and we propose a novel multi-agent constrained soft actor-critic (MACSAC) reinforcement learning algorithm. MACSAC is used to train the control agents online, so the accurate ADN model is no longer needed. Then, the trained agents can realize decentralized optimal control using local measurements without real-time P2P communication. The OLDC with MACSAC has shown extraordinary flexibility, efficiency and robustness to various computing and communication conditions. Numerical simulations on IEEE test cases not only demonstrate that the proposed MACSAC outperforms the state-of-art learning algorithms, but also support the superiority of our OLDC framework in the online application.

READ FULL TEXT
research
07/06/2020

Consensus Multi-Agent Reinforcement Learning for Volt-VAR Control in Power Distribution Networks

Volt-VAR control (VVC) is a critical application in active distribution ...
research
05/20/2020

Two-stage Deep Reinforcement Learning for Inverter-based Volt-VAR Control in Active Distribution Networks

Model-based Vol/VAR optimization method is widely used to eliminate volt...
research
02/16/2021

Multi-Stage Transmission Line Flow Control Using Centralized and Decentralized Reinforcement Learning Agents

Planning future operational scenarios of bulk power systems that meet se...
research
06/07/2020

Real-Time Model Calibration with Deep Reinforcement Learning

The dynamic, real-time, and accurate inference of model parameters from ...
research
08/16/2021

Decentralized Multi-AGV Task Allocation based on Multi-Agent Reinforcement Learning with Information Potential Field Rewards

Automated Guided Vehicles (AGVs) have been widely used for material hand...
research
05/22/2023

Risk-aware Safe Control for Decentralized Multi-agent Systems via Dynamic Responsibility Allocation

Decentralized control schemes are increasingly favored in various domain...

Please sign up or login with your details

Forgot password? Click here to reset