Scalable Multiagent Coordination with Distributed Online Open Loop Planning

02/24/2017
by   Lenz Belzner, et al.
0

We propose distributed online open loop planning (DOOLP), a general framework for online multiagent coordination and decision making under uncertainty. DOOLP is based on online heuristic search in the space defined by a generative model of the domain dynamics, which is exploited by agents to simulate and evaluate the consequences of their potential choices. We also propose distributed online Thompson sampling (DOTS) as an effective instantiation of the DOOLP framework. DOTS models sequences of agent choices by concatenating a number of multiarmed bandits for each agent and uses Thompson sampling for dealing with action value uncertainty. The Bayesian approach underlying Thompson sampling allows to effectively model and estimate uncertainty about (a) own action values and (b) other agents' behavior. This approach yields a principled and statistically sound solution to the exploration-exploitation dilemma when exploring large search spaces with limited resources. We implemented DOTS in a smart factory case study with positive empirical results. We observed effective, robust and scalable planning and coordination capabilities even when only searching a fraction of the potential search space.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/12/2021

Scalable Anytime Planning for Multi-Agent MDPs

We present a scalable tree search planning algorithm for large multi-age...
research
04/17/2018

Leveraging Statistical Multi-Agent Online Planning with Emergent Value Function Approximation

Making decisions is a great challenge in distributed autonomous environm...
research
07/11/2019

Adaptive Thompson Sampling Stacks for Memory Bounded Open-Loop Planning

We propose Stable Yet Memory Bounded Open-Loop (SYMBOL) planning, a gene...
research
09/01/2023

Implementing BDI Continual Temporal Planning for Robotic Agents

Making autonomous agents effective in real-life applications requires th...
research
04/04/2014

Scalable Planning and Learning for Multiagent POMDPs: Extended Version

Online, sample-based planning algorithms for POMDPs have shown great pro...
research
04/18/2020

Remote Empirical Coordination

We apply the framework of imperfect empirical coordination to a two-node...
research
04/01/2023

Factorization of Multi-Agent Sampling-Based Motion Planning

Modern robotics often involves multiple embodied agents operating within...

Please sign up or login with your details

Forgot password? Click here to reset