Scalable Reinforcement Learning Policies for Multi-Agent Control

by   Christopher D. Hsu, et al.

This paper develops a stochastic Multi-Agent Reinforcement Learning (MARL) method to learn control policies that can handle an arbitrary number of external agents; our policies can be executed for tasks consisting of 1000 pursuers and 1000 evaders. We model pursuers as agents with limited on-board sensing and formulate the problem as a decentralized, partially-observable Markov Decision Process. An attention mechanism is used to build a permutation and input-size invariant embedding of the observations for learning a stochastic policy and value function using techniques in entropy-regularized off-policy methods. Simulation experiments on a large number of problems show that our control policies are dramatically scalable and display cooperative behavior in spite of being executed in a decentralized fashion; our methods offer a simple solution to classical multi-agent problems using techniques in reinforcement learning.


page 1

page 7


Scalable Centralized Deep Multi-Agent Reinforcement Learning via Policy Gradients

In this paper, we explore using deep reinforcement learning for problems...

Cooperative multi-agent reinforcement learning for high-dimensional nonequilibrium control

Experimental advances enabling high-resolution external control create n...

Q-Mixing Network for Multi-Agent Pathfinding in Partially Observable Grid Environments

In this paper, we consider the problem of multi-agent navigation in part...

Frequency-Based Patrolling with Heterogeneous Agents and Limited Communication

This paper investigates multi-agent frequencybased patrolling of interse...

IntelligentCrowd: Mobile Crowdsensing via Multi-agent Reinforcement Learning

The prosperity of smart mobile devices has made mobile crowdsensing (MCS...

Information State Embedding in Partially Observable Cooperative Multi-Agent Reinforcement Learning

Multi-agent reinforcement learning (MARL) under partial observability ha...

Common Information based Approximate State Representations in Multi-Agent Reinforcement Learning

Due to information asymmetry, finding optimal policies for Decentralized...