Neural Tree Expansion for Multi-Robot Planning in Non-Cooperative Environments

by   Benjamin Rivière, et al.

We present a self-improving, neural tree expansion method for multi-robot online planning in non-cooperative environments, where each robot tries to maximize its cumulative reward while interacting with other self-interested robots. Our algorithm adapts the centralized, perfect information, discrete-action space method from Alpha Zero to a decentralized, partial information, continuous action space setting for multi-robot applications. Our method has three interacting components: (i) a centralized, perfect-information `expert' Monte Carlo Tree Search (MCTS) with large computation resources that provides expert demonstrations, (ii) a decentralized, partial-information `learner' MCTS with small computation resources that runs in real-time and provides self-play examples, and (iii) policy value neural networks that are trained with the expert demonstrations and bias both the expert and the learner tree growth. Our numerical experiments demonstrate neural expansion generates compact search trees with better solution quality and 20 times less computational expense compared to MCTS without neural expansion. The resulting policies are dynamically sophisticated, demonstrate coordination between robots, and play the Reach-Target-Avoid differential game significantly better than the state-of-the-art control-theoretic baseline for multi-robot, double-integrator systems. Our hardware experiments on an aerial swarm demonstrate the computational advantage of neural tree expansion, enabling online planning at 20Hz with effective policies in complex scenarios.



There are no comments yet.


page 1


Learning Cooperation and Online Planning Through Simulation and Graph Convolutional Network

Multi-agent Markov Decision Process (MMDP) has been an effective way of ...

Policy Gradient Search: Online Planning and Expert Iteration without Search Trees

Monte Carlo Tree Search (MCTS) algorithms perform simulation-based searc...

GLAS: Global-to-Local Safe Autonomy Synthesis for Multi-Robot Motion Planning with End-to-End Learning

We present GLAS: Global-to-Local Autonomy Synthesis, a provably-safe, au...

Decentralized Cooperative Planning for Automated Vehicles with Continuous Monte Carlo Tree Search

Urban traffic scenarios often require a high degree of cooperation betwe...

Q-CP: Learning Action Values for Cooperative Planning

Research on multi-robot systems has demonstrated promising results in ma...

Decentralized Connectivity-Preserving Deployment of Large-Scale Robot Swarms

We present a decentralized and scalable approach for deployment of a rob...

Data-Efficient Learning of High-Quality Controls for Kinodynamic Planning used in Vehicular Navigation

This paper aims to improve the path quality and computational efficiency...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.