DeepAI AI Chat
Log In Sign Up

Iterative Update and Unified Representation for Multi-Agent Reinforcement Learning

08/16/2019
by   Jiancheng Long, et al.
Nanchang University
ia.ac.cn
0

Multi-agent systems have a wide range of applications in cooperative and competitive tasks. As the number of agents increases, nonstationarity gets more serious in multi-agent reinforcement learning (MARL), which brings great difficulties to the learning process. Besides, current mainstream algorithms configure each agent an independent network,so that the memory usage increases linearly with the number of agents which greatly slows down the interaction with the environment. Inspired by Generative Adversarial Networks (GAN), this paper proposes an iterative update method (IU) to stabilize the nonstationary environment. Further, we add first-person perspective and represent all agents by only one network which can change agents' policies from sequential compute to batch compute. Similar to continual lifelong learning, we realize the iterative update method in this unified representative network (IUUR). In this method, iterative update can greatly alleviate the nonstationarity of the environment, unified representation can speed up the interaction with environment and avoid the linear growth of memory usage. Besides, this method does not bother decentralized execution and distributed deployment. Experiments show that compared with MADDPG, our algorithm achieves state-of-the-art performance and saves wall-clock time by a large margin especially with more agents.

READ FULL TEXT
02/20/2022

PooL: Pheromone-inspired Communication Framework forLarge Scale Multi-Agent Reinforcement Learning

Being difficult to scale poses great problems in multi-agent coordinatio...
09/13/2017

Learning with Opponent-Learning Awareness

Multi-agent settings are quickly gathering importance in machine learnin...
05/14/2022

Unified Distributed Environment

We propose Unified Distributed Environment (UDE), an environment virtual...
11/11/2019

SMIX(λ): Enhancing Centralized Value Functions for Cooperative Multi-Agent Reinforcement Learning

Learning a stable and generalizable centralized value function (CVF) is ...
04/14/2022

Usage-based learning of grammatical categories

Human languages use a wide range of grammatical categories to constrain ...
08/10/2020

Distributed Multi-agent Video Fast-forwarding

In many intelligent systems, a network of agents collaboratively perceiv...