PowerNet: Multi-agent Deep Reinforcement Learning for Scalable Powergrid Control

11/24/2020
by   Dong Chen, et al.
0

This paper develops an efficient multi-agent deep reinforcement learning algorithm for cooperative controls in powergrids. Specifically, we consider the decentralized inverter-based secondary voltage control problem in distributed generators (DGs), which is first formulated as a cooperative multi-agent reinforcement learning (MARL) problem. We then propose a novel on-policy MARL algorithm, PowerNet, in which each agent (DG) learns a control policy based on (sub-)global reward but local states from its neighboring agents. Motivated by the fact that a local control from one agent has limited impact on agents distant from it, we exploit a novel spatial discount factor to reduce the effect from remote agents, to expedite the training process and improve scalability. Furthermore, a differentiable, learning-based communication protocol is employed to foster the collaborations among neighboring agents. In addition, to mitigate the effects of system uncertainty and random noise introduced during on-policy learning, we utilize an action smoothing factor to stabilize the policy execution. To facilitate training and evaluation, we develop PGSim, an efficient, high-fidelity powergrid simulation platform. Experimental results in two microgrid setups show that the developed PowerNet outperforms a conventional model-based control, as well as several state-of-the-art MARL algorithms. The decentralized learning scheme and high sample efficiency also make it viable to large-scale power grids.

READ FULL TEXT

page 1

page 4

research
04/03/2020

Multi-agent Reinforcement Learning for Networked System Control

This paper considers multi-agent reinforcement learning (MARL) in networ...
research
12/15/2022

Distributed-Training-and-Execution Multi-Agent Reinforcement Learning for Power Control in HetNet

In heterogeneous networks (HetNets), the overlap of small cells and the ...
research
11/02/2020

Cooperative Heterogeneous Deep Reinforcement Learning

Numerous deep reinforcement learning agents have been proposed, and each...
research
10/16/2020

Decentralized Multi-Agent Pursuit using Deep Reinforcement Learning

Pursuit-evasion is the problem of capturing mobile targets with one or m...
research
05/31/2020

Distributed Voltage Regulation of Active Distribution System Based on Enhanced Multi-agent Deep Reinforcement Learning

This paper proposes a data-driven distributed voltage control approach b...
research
11/05/2019

Learning to flock through reinforcement

Flocks of birds, schools of fish, insects swarms are examples of coordin...
research
02/24/2023

Multi-Agent Reinforcement Learning with Common Policy for Antenna Tilt Optimization

This paper proposes a method for wireless network optimization applicabl...

Please sign up or login with your details

Forgot password? Click here to reset