Scalable Anytime Planning for Multi-Agent MDPs

by   Shushman Choudhury, et al.

We present a scalable tree search planning algorithm for large multi-agent sequential decision problems that require dynamic collaboration. Teams of agents need to coordinate decisions in many domains, but naive approaches fail due to the exponential growth of the joint action space with the number of agents. We circumvent this complexity through an anytime approach that allows us to trade computation for approximation quality and also dynamically coordinate actions. Our algorithm comprises three elements: online planning with Monte Carlo Tree Search (MCTS), factored representations of local agent interactions with coordination graphs, and the iterative Max-Plus method for joint action selection. We evaluate our approach on the benchmark SysAdmin domain with static coordination graphs and achieve comparable performance with much lower computation cost than our MCTS baselines. We also introduce a multi-drone delivery domain with dynamic, i.e., state-dependent coordination graphs, and demonstrate how our approach scales to large problems on this domain that are intractable for other MCTS methods. We provide an open-source implementation of our algorithm at



There are no comments yet.


page 1

page 2

page 3

page 4


Learning Cooperation and Online Planning Through Simulation and Graph Convolutional Network

Multi-agent Markov Decision Process (MMDP) has been an effective way of ...

Scalable Decision-Theoretic Planning in Open and Typed Multiagent Systems

In open agent systems, the set of agents that are cooperating or competi...

Scalable Multiagent Coordination with Distributed Online Open Loop Planning

We propose distributed online open loop planning (DOOLP), a general fram...

Thompson Sampling for Pursuit-Evasion Problems

Pursuit-evasion is a multi-agent sequential decision problem wherein a g...

Dynamic Multi-Robot Task Allocation under Uncertainty and Temporal Constraints

We consider the problem of dynamically allocating tasks to multiple agen...

A Cordial Sync: Going Beyond Marginal Policies for Multi-Agent Embodied Tasks

Autonomous agents must learn to collaborate. It is not scalable to devel...

X*: Anytime Multiagent Planning With Bounded Search

Multi-agent planning in dynamic domains is a challenging problem: the si...

Code Repositories


Scalable MCTS for team scenarios

view repo
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.