REMAX: Relational Representation for Multi-Agent Exploration

08/12/2020
by   Heechang Ryu, et al.
0

Training a multi-agent reinforcement learning (MARL) model is generally difficult because there are numerous combinations of complex interactions among agents that induce certain reward signals. Especially when there is a sparse reward signal, the training becomes more difficult. Previous studies have tried to resolve this issue by employing an intrinsic reward, which is a signal specifically designed for inducing the interactions among agents, to boost the MARL model training. However, this approach requires extensive prior knowledge to design an intrinsic reward. To optimize the training of an MARL model, we propose a learning-based exploration strategy to generate the initial states of a game. The proposed method adopts a variational graph autoencoder to represent a state of a game such that (1) the state can be compactly encoded to the latent representation by considering the relationship among agents, and (2) the latent representation can be used as an effective input to the surrogate model predicting the exploration score. The proposed method determines the latent representations that maximize the surrogate model and decodes these representations to generate the initial states from which the MARL model starts training. Empirically, we demonstrate that the generated states improve the training and performance of MARL more than the existing exploration methods.

READ FULL TEXT
research
10/12/2019

Influence-Based Multi-Agent Exploration

Intrinsically motivated reinforcement learning aims to address the explo...
research
10/29/2022

Curiosity-Driven Multi-Agent Exploration with Mixed Objectives

Intrinsic rewards have been increasingly used to mitigate the sparse rew...
research
06/10/2020

The Emergence of Individuality in Multi-Agent Reinforcement Learning

Individuality is essential in human society, which induces the division ...
research
02/19/2023

AIIR-MIX: Multi-Agent Reinforcement Learning Meets Attention Individual Intrinsic Reward Mixing Network

Deducing the contribution of each agent and assigning the corresponding ...
research
04/21/2019

Generative Exploration and Exploitation

Sparse reward is one of the biggest challenges in reinforcement learning...
research
06/01/2018

Deep Curiosity Search: Intra-Life Exploration Improves Performance on Challenging Deep Reinforcement Learning Problems

Traditional exploration methods in RL require agents to perform random a...
research
10/24/2022

IDRL: Identifying Identities in Multi-Agent Reinforcement Learning with Ambiguous Identities

Multi-agent reinforcement learning(MARL) is a prevalent learning paradig...

Please sign up or login with your details

Forgot password? Click here to reset