A Learning-based Optimal Market Bidding Strategy for Price-Maker Energy Storage

06/04/2021
by   Mathilde D. Badoual, et al.
0

Load serving entities with storage units reach sizes and performances that can significantly impact clearing prices in electricity markets. Nevertheless, price endogeneity is rarely considered in storage bidding strategies and modeling the electricity market is a challenging task. Meanwhile, model-free reinforcement learning such as the Actor-Critic are becoming increasingly popular for designing energy system controllers. Yet implementation frequently requires lengthy, data-intense, and unsafe trial-and-error training. To fill these gaps, we implement an online Supervised Actor-Critic (SAC) algorithm, supervised with a model-based controller – Model Predictive Control (MPC). The energy storage agent is trained with this algorithm to optimally bid while learning and adjusting to its impact on the market clearing prices. We compare the supervised Actor-Critic algorithm with the MPC algorithm as a supervisor, finding that the former reaps higher profits via learning. Our contribution, thus, is an online and safe SAC algorithm that outperforms the current model-based state-of-the-art.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 6

04/29/2020

How to Learn a Useful Critic? Model-based Action-Gradient-Estimator Policy Optimization

Deterministic-policy actor-critic algorithms for continuous control impr...
12/09/2020

Deep Reinforcement Learning for Long Term Hydropower Production Scheduling

We explore the use of deep reinforcement learning to provide strategies ...
10/04/2020

FORK: A Forward-Looking Actor For Model-Free Reinforcement Learning

In this paper, we propose a new type of Actor, named forward-looking Act...
04/03/2020

Reinforcement Learning for Mixed-Integer Problems Based on MPC

Model Predictive Control has been recently proposed as policy approximat...
04/04/2020

Model-based actor-critic: GAN + DRL (actor-critic) => AGI

Our effort is toward unifying GAN and DRL algorithms into a unifying AI ...
02/28/2019

Infer Your Enemies and Know Yourself, Learning in Real-Time Bidding with Partially Observable Opponents

Real-time bidding, as one of the most popular mechanisms for selling onl...
01/14/2019

Online Inventory Management with Application to Energy Procurement in Data Centers

Motivated by the application of energy storage management in electricity...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.