DeepAI AI Chat
Log In Sign Up

Approximating Energy Market Clearing and Bidding With Model-Based Reinforcement Learning

03/03/2023
by   Thomas Wolgast, et al.
0

Energy markets can provide incentives for undesired behavior of market participants. Multi-agent Reinforcement learning (MARL) is a promising new approach to determine the expected behavior of energy market participants. However, reinforcement learning requires many interactions with the system to converge, and the power system environment often consists of extensive computations, e.g., optimal power flow (OPF) calculation for market clearing. To tackle this complexity, we provide a model of the energy market to a basic MARL algorithm, in form of a learned OPF approximation and explicit market rules. The learned OPF surrogate model makes an explicit solving of the OPF completely unnecessary. Our experiments demonstrate that the model additionally reduces training time by about one order of magnitude, but at the cost of a slightly worse approximation of the Nash equilibrium. Potential applications of our method are market design, more realistic modeling of market participants, and analysis of manipulative behavior.

READ FULL TEXT
09/23/2020

A Multi-Agent Deep Reinforcement Learning Approach for a Distributed Energy Marketplace in Smart Grids

This paper presents a Reinforcement Learning (RL) based energy market fo...
06/10/2021

Achieving Diverse Objectives with AI-driven Prices in Deep Reinforcement Learning Multi-agent Markets

We propose a practical approach to computing market prices and allocatio...
05/14/2019

A Reputation System for Multi-Agent Marketplaces

We present an exploration of a reputation system based on explicit ratin...
03/03/2020

Robust Market Making via Adversarial Reinforcement Learning

We show that adversarial reinforcement learning (ARL) can be used to pro...
02/12/2022

Applying multi product lines to equity market software ecosystem

Context: In recent decades, many financial markets and their participant...
10/29/2021

Profit equitably: An investigation of market maker's impact on equitable outcomes

We look at discovering the impact of market microstructure on equitabili...
06/09/2011

Accelerating Reinforcement Learning by Composing Solutions of Automatically Identified Subtasks

This paper discusses a system that accelerates reinforcement learning by...