Log In Sign Up

Multi-Agent Reinforcement Learning for Persistent Monitoring

by   Jingxi Chen, et al.

The Persistent Monitoring (PM) problem seeks to find a set of trajectories (or controllers) for robots to persistently monitor a changing environment. Each robot has a limited field-of-view and may need to coordinate with others to ensure no point in the environment is left unmonitored for long periods of time. We model the problem such that there is a penalty that accrues every time step if a point is left unmonitored. However, the dynamics of the penalty are unknown to us. We present a Multi-Agent Reinforcement Learning (MARL) algorithm for the persistent monitoring problem. Specifically, we present a Multi-Agent Graph Attention Proximal Policy Optimization (MA-G-PPO) algorithm that takes as input the local observations of all agents combined with a low resolution global map to learn a policy for each agent. The graph attention allows agents to share their information with others leading to an effective joint policy. Our main focus is to understand how effective MARL is for the PM problem. We investigate five research questions with this broader goal. We find that MA-G-PPO is able to learn a better policy than the non-RL baseline in most cases, the effectiveness depends on agents sharing information with each other, and the policy learnt shows emergent behavior for the agents.


page 2

page 4

page 6


GALOPP: Multi-Agent Deep Reinforcement Learning For Persistent Monitoring With Localization Constraints

Persistently monitoring a region under localization and communication co...

Efficient Domain Coverage for Vehicles with Second Order Dynamics via Multi-Agent Reinforcement Learning

Collaborative autonomous multi-agent systems covering a specified area h...

Consolidation via Policy Information Regularization in Deep RL for Multi-Agent Games

This paper introduces an information-theoretic constraint on learned pol...

Deep Reinforcement Learning Based Multi-Access Edge Computing Schedule for Internet of Vehicle

As intelligent transportation systems been implemented broadly and unman...

A sub-modular receding horizon solution for mobile multi-agent persistent monitoring

We study the problem of persistent monitoring of finite number of inter-...

Attention-based Fault-tolerant Approach for Multi-agent Reinforcement Learning Systems

The aim of multi-agent reinforcement learning systems is to provide inte...

DM^2: Distributed Multi-Agent Reinforcement Learning for Distribution Matching

Current approaches to multi-agent cooperation rely heavily on centralize...