Multi-Robot Deep Reinforcement Learning with Macro-Actions

09/19/2019
by   Yuchen Xiao, et al.
0

In many real-world multi-robot tasks, high-quality solutions often require a team of robots to perform asynchronous actions under decentralized control. Multi-agent reinforcement learning methods have difficulty learning decentralized policies because the environment appearing to be non-stationary due to other agents also learning at the same time. In this paper, we address this challenge by proposing a macro-action-based decentralized multi-agent double deep recurrent Q-net (MacDec-MADDRQN) which creates a new double Q-updating rule to train each decentralized Q-net using a centralized Q-net for action selection. A generalized version of MacDec-MADDRQN with two separate training environments, called Parallel-MacDec-MADDRQN, is also presented to cope with the uncertainty in adopting either centralized or decentralized exploration. The advantages and the practical nature of our methods are demonstrated by achieving near-centralized results in simulation experiments and permitting real robots to accomplish a warehouse tool delivery task in an efficient way.

READ FULL TEXT

page 1

page 6

research
09/20/2022

Macro-Action-Based Multi-Agent/Robot Deep Reinforcement Learning under Partial Observability

The state-of-the-art multi-agent reinforcement learning (MARL) methods h...
research
10/05/2021

Deep Reinforcement Learning for Decentralized Multi-Robot Exploration with Macro Actions

Cooperative multi-robot teams need to be able to explore cluttered and u...
research
04/18/2020

Macro-Action-Based Deep Multi-Agent Reinforcement Learning

In real-world multi-robot systems, performing high-quality, collaborativ...
research
07/03/2019

End-to-end Decentralized Multi-robot Navigation in Unknown Complex Environments via Deep Reinforcement Learning

In this paper, a novel deep reinforcement learning (DRL)-based method is...
research
01/09/2023

Asynchronous Multi-Agent Reinforcement Learning for Efficient Real-Time Multi-Robot Cooperative Exploration

We consider the problem of cooperative exploration where multiple robots...
research
12/16/2021

Centralizing State-Values in Dueling Networks for Multi-Robot Reinforcement Learning Mapless Navigation

We study the problem of multi-robot mapless navigation in the popular Ce...

Please sign up or login with your details

Forgot password? Click here to reset