Enforcing Policy Feasibility Constraints through Differentiable Projection for Energy Optimization

05/19/2021
by   Bingqing Chen, et al.
0

While reinforcement learning (RL) is gaining popularity in energy systems control, its real-world applications are limited due to the fact that the actions from learned policies may not satisfy functional requirements or be feasible for the underlying physical system. In this work, we propose PROjected Feasibility (PROF), a method to enforce convex operational constraints within neural policies. Specifically, we incorporate a differentiable projection layer within a neural network-based policy to enforce that all learned actions are feasible. We then update the policy end-to-end by propagating gradients through this differentiable projection layer, making the policy cognizant of the operational constraints. We demonstrate our method on two applications: energy-efficient building operation and inverter control. In the building operation setting, we show that PROF maintains thermal comfort requirements while improving energy efficiency by 4 inverter control setting, PROF perfectly satisfies voltage constraints on the IEEE 37-bus feeder system, as it learns to curtail as little renewable energy as possible within its safety set.

READ FULL TEXT

page 2

page 7

research
09/16/2022

Optimizing Industrial HVAC Systems with Hierarchical Reinforcement Learning

Reinforcement learning (RL) techniques have been developed to optimize i...
research
10/23/2022

MetaEMS: A Meta Reinforcement Learning-based Control Framework for Building Energy Management System

The building sector has been recognized as one of the primary sectors fo...
research
06/13/2023

Multi-market Energy Optimization with Renewables via Reinforcement Learning

This paper introduces a deep reinforcement learning (RL) framework for o...
research
06/22/2020

Non-convex Optimization via Adaptive Stochastic Search for End-to-End Learning and Control

In this work we propose the use of adaptive stochastic search as a build...
research
06/20/2020

Accelerating Safe Reinforcement Learning with Constraint-mismatched Policies

We consider the problem of reinforcement learning when provided with a b...
research
02/27/2022

Neural-Progressive Hedging: Enforcing Constraints in Reinforcement Learning with Stochastic Programming

We propose a framework, called neural-progressive hedging (NP), that lev...
research
03/22/2021

DeepOPF-V: Solving AC-OPF Problems Efficiently

AC optimal power flow (AC-OPF) problems need to be solved more frequentl...

Please sign up or login with your details

Forgot password? Click here to reset