DeepAI AI Chat
Log In Sign Up

One-Step Two-Critic Deep Reinforcement Learning for Inverter-based Volt-Var Control in Active Distribution Networks

by   Qiong Liu, et al.

A one-step two-critic deep reinforcement learning (OSTC-DRL) approach for inverter-based volt-var control (IB-VVC) in active distribution networks is proposed in this paper. Firstly, considering IB-VVC can be formulated as a single-period optimization problem, we formulate the IB-VVC as a one-step Markov decision process rather than the standard Markov decision process, which simplifies the DRL learning task. Then we design the one-step actor-critic DRL scheme which is a simplified version of recent DRL algorithms, and it avoids the issue of Q value overestimation successfully. Furthermore, considering two objectives of VVC: minimizing power loss and eliminating voltage violation, we utilize two critics to approximate the rewards of two objectives separately. It simplifies the approximation tasks of each critic, and avoids the interaction effect between two objectives in the learning process of critic. The OSTC-DRL approach integrates the one-step actor-critic DRL scheme and the two-critic technology. Based on the OSTC-DRL, we design two centralized DRL algorithms. Further, we extend the OSTC-DRL to multi-agent OSTC-DRL for decentralized IB-VVC and design two multi-agent DRL algorithms. Simulations demonstrate that the proposed OSTC-DRL has a faster convergence rate and a better control performance, and the multi-agent OSTC-DRL works well for decentralized IB-VVC problems.


page 1

page 9


Centralized Distributed Deep Reinforcement Learning Methods for Downlink Sum-Rate Optimization

For a multi-cell, multi-user, cellular network downlink sum-rate maximiz...

Actor-Critic Reinforcement Learning for Control with Stability Guarantee

Deep Reinforcement Learning (DRL) has achieved impressive performance in...

Managing engineering systems with large state and action spaces through deep reinforcement learning

Decision-making for engineering systems can be efficiently formulated as...

A Human Mixed Strategy Approach to Deep Reinforcement Learning

In 2015, Google's DeepMind announced an advancement in creating an auton...

Semantic and Effective Communication for Remote Control Tasks with Dynamic Feature Compression

The coordination of robotic swarms and the remote wireless control of in...

Online 3D Bin Packing with Constrained Deep Reinforcement Learning

We solve a challenging yet practically useful variant of 3D Bin Packing ...