Scalable Voltage Control using Structure-Driven Hierarchical Deep Reinforcement Learning

01/29/2021
by   Sayak Mukherjee, et al.
0

This paper presents a novel hierarchical deep reinforcement learning (DRL) based design for the voltage control of power grids. DRL agents are trained for fast, and adaptive selection of control actions such that the voltage recovery criterion can be met following disturbances. Existing voltage control techniques suffer from the issues of speed of operation, optimal coordination between different locations, and scalability. We exploit the area-wise division structure of the power system to propose a hierarchical DRL design that can be scaled to the larger grid models. We employ an enhanced augmented random search algorithm that is tailored for the voltage control problem in a two-level architecture. We train area-wise decentralized RL agents to compute lower-level policies for the individual areas, and concurrently train a higher-level DRL agent that uses the updates of the lower-level policies to efficiently coordinate the control actions taken by the lower-level agents. Numerical experiments on the IEEE benchmark 39-bus model with 3 areas demonstrate the advantages and various intricacies of the proposed hierarchical approach.

READ FULL TEXT
research
06/25/2022

Hierarchical Reinforcement Learning with Opponent Modeling for Distributed Multi-agent Cooperation

Many real-world applications can be formulated as multi-agent cooperatio...
research
10/14/2019

Coordination of PV Smart Inverters Using Deep Reinforcement Learning for Grid Voltage Regulation

Increasing adoption of solar photovoltaic (PV) presents new challenges t...
research
12/23/2020

Rethink AI-based Power Grid Control: Diving Into Algorithm Design

Recently, deep reinforcement learning (DRL)-based approach has shown pro...
research
07/31/2023

Distributionally Robust Safety Filter for Learning-Based Control in Active Distribution Systems

Operational constraint violations may occur when deep reinforcement lear...
research
01/13/2021

Learning and Fast Adaptation for Grid Emergency Control via Deep Meta Reinforcement Learning

As power systems are undergoing a significant transformation with more u...
research
09/29/2022

Hierarchical Training of Deep Ensemble Policies for Reinforcement Learning in Continuous Spaces

Many actor-critic deep reinforcement learning (DRL) algorithms have achi...
research
09/18/2020

RLzoo: A Comprehensive and Adaptive Reinforcement Learning Library

Recently, we have seen a rapidly growing adoption of Deep Reinforcement ...

Please sign up or login with your details

Forgot password? Click here to reset