Neural Tree Expansion for Multi-Robot Planning in Non-Cooperative Environments

04/20/2021
by   Benjamin Rivière, et al.
0

We present a self-improving, neural tree expansion method for multi-robot online planning in non-cooperative environments, where each robot tries to maximize its cumulative reward while interacting with other self-interested robots. Our algorithm adapts the centralized, perfect information, discrete-action space method from Alpha Zero to a decentralized, partial information, continuous action space setting for multi-robot applications. Our method has three interacting components: (i) a centralized, perfect-information `expert' Monte Carlo Tree Search (MCTS) with large computation resources that provides expert demonstrations, (ii) a decentralized, partial-information `learner' MCTS with small computation resources that runs in real-time and provides self-play examples, and (iii) policy value neural networks that are trained with the expert demonstrations and bias both the expert and the learner tree growth. Our numerical experiments demonstrate neural expansion generates compact search trees with better solution quality and 20 times less computational expense compared to MCTS without neural expansion. The resulting policies are dynamically sophisticated, demonstrate coordination between robots, and play the Reach-Target-Avoid differential game significantly better than the state-of-the-art control-theoretic baseline for multi-robot, double-integrator systems. Our hardware experiments on an aerial swarm demonstrate the computational advantage of neural tree expansion, enabling online planning at 20Hz with effective policies in complex scenarios.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

10/16/2021

Learning Cooperation and Online Planning Through Simulation and Graph Convolutional Network

Multi-agent Markov Decision Process (MMDP) has been an effective way of ...
04/07/2019

Policy Gradient Search: Online Planning and Expert Iteration without Search Trees

Monte Carlo Tree Search (MCTS) algorithms perform simulation-based searc...
02/26/2020

GLAS: Global-to-Local Safe Autonomy Synthesis for Multi-Robot Motion Planning with End-to-End Learning

We present GLAS: Global-to-Local Autonomy Synthesis, a provably-safe, au...
09/10/2018

Decentralized Cooperative Planning for Automated Vehicles with Continuous Monte Carlo Tree Search

Urban traffic scenarios often require a high degree of cooperation betwe...
03/01/2018

Q-CP: Learning Action Values for Cooperative Planning

Research on multi-robot systems has demonstrated promising results in ma...
06/01/2018

Decentralized Connectivity-Preserving Deployment of Large-Scale Robot Swarms

We present a decentralized and scalable approach for deployment of a rob...
01/06/2022

Data-Efficient Learning of High-Quality Controls for Kinodynamic Planning used in Vehicular Navigation

This paper aims to improve the path quality and computational efficiency...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.