Room Clearance with Feudal Hierarchical Reinforcement Learning

by   Henry Charlesworth, et al.

Reinforcement learning (RL) is a general framework that allows systems to learn autonomously through trial-and-error interaction with their environment. In recent years combining RL with expressive, high-capacity neural network models has led to impressive performance in a diverse range of domains. However, dealing with the large state and action spaces often required for problems in the real world still remains a significant challenge. In this paper we introduce a new simulation environment, "Gambit", designed as a tool to build scenarios that can drive RL research in a direction useful for military analysis. Using this environment we focus on an abstracted and simplified room clearance scenario, where a team of blue agents have to make their way through a building and ensure that all rooms are cleared of (and remain clear) of enemy red agents. We implement a multi-agent version of feudal hierarchical RL that introduces a command hierarchy where a commander at the higher level sends orders to multiple agents at the lower level who simply have to learn to follow these orders. We find that breaking the task down in this way allows us to solve a number of non-trivial floorplans that require the coordination of multiple agents much more efficiently than the standard baseline RL algorithms we compare with. We then go on to explore how qualitatively different behaviour can emerge depending on what we prioritise in the agent's reward function (e.g. clearing the building quickly vs. prioritising rescuing civilians).



There are no comments yet.


page 1

page 3

page 5

page 6

page 7


Extended Markov Games to Learn Multiple Tasks in Multi-Agent Reinforcement Learning

The combination of Formal Methods with Reinforcement Learning (RL) has r...

FireCommander: An Interactive, Probabilistic Multi-agent Environment for Joint Perception-Action Tasks

The purpose of this tutorial is to help individuals use the FireCommande...

Hierarchical Program-Triggered Reinforcement Learning Agents For Automated Driving

Recent advances in Reinforcement Learning (RL) combined with Deep Learni...

Multi-Agent Transfer Learning in Reinforcement Learning-Based Ride-Sharing Systems

Reinforcement learning (RL) has been used in a range of simulated real-w...

Simplified Action Decoder for Deep Multi-Agent Reinforcement Learning

In recent years we have seen fast progress on a number of benchmark prob...

Learning how to Interact with a Complex Interface using Hierarchical Reinforcement Learning

Hierarchical Reinforcement Learning (HRL) allows interactive agents to d...

Theory-based Causal Transfer: Integrating Instance-level Induction and Abstract-level Structure Learning

Learning transferable knowledge across similar but different settings is...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.