DeepAI AI Chat
Log In Sign Up

Approximating Energy Market Clearing and Bidding With Model-Based Reinforcement Learning

by   Thomas Wolgast, et al.

Energy markets can provide incentives for undesired behavior of market participants. Multi-agent Reinforcement learning (MARL) is a promising new approach to determine the expected behavior of energy market participants. However, reinforcement learning requires many interactions with the system to converge, and the power system environment often consists of extensive computations, e.g., optimal power flow (OPF) calculation for market clearing. To tackle this complexity, we provide a model of the energy market to a basic MARL algorithm, in form of a learned OPF approximation and explicit market rules. The learned OPF surrogate model makes an explicit solving of the OPF completely unnecessary. Our experiments demonstrate that the model additionally reduces training time by about one order of magnitude, but at the cost of a slightly worse approximation of the Nash equilibrium. Potential applications of our method are market design, more realistic modeling of market participants, and analysis of manipulative behavior.


A Multi-Agent Deep Reinforcement Learning Approach for a Distributed Energy Marketplace in Smart Grids

This paper presents a Reinforcement Learning (RL) based energy market fo...

Achieving Diverse Objectives with AI-driven Prices in Deep Reinforcement Learning Multi-agent Markets

We propose a practical approach to computing market prices and allocatio...

A Reputation System for Multi-Agent Marketplaces

We present an exploration of a reputation system based on explicit ratin...

Robust Market Making via Adversarial Reinforcement Learning

We show that adversarial reinforcement learning (ARL) can be used to pro...

Applying multi product lines to equity market software ecosystem

Context: In recent decades, many financial markets and their participant...

Profit equitably: An investigation of market maker's impact on equitable outcomes

We look at discovering the impact of market microstructure on equitabili...

Accelerating Reinforcement Learning by Composing Solutions of Automatically Identified Subtasks

This paper discusses a system that accelerates reinforcement learning by...