The Complexity of Decentralized Control of Markov Decision Processes

01/16/2013
by   Daniel S. Bernstein, et al.
0

Planning for distributed agents with partial state information is considered from a decision- theoretic perspective. We describe generalizations of both the MDP and POMDP models that allow for decentralized control. For even a small number of agents, the finite-horizon problems corresponding to both of our models are complete for nondeterministic exponential time. These complexity results illustrate a fundamental difference between centralized and decentralized control of Markov processes. In contrast to the MDP and POMDP problems, the problems we consider provably do not admit polynomial-time algorithms and most likely require doubly exponential time to solve in the worst case. We have thus provided mathematical evidence corresponding to the intuition that decentralized planning problems cannot easily be reduced to centralized problems and solved exactly using established techniques.

READ FULL TEXT

page 1

page 2

page 3

page 5

06/30/2011

Decentralized Control of Cooperative Systems: Categorization and Complexity Analysis

Decentralized control of cooperative systems captures the operation of a...
03/24/2022

Horizon-Free Reinforcement Learning in Polynomial Time: the Power of Stationary Policies

This paper gives the first polynomial-time algorithm for tabular Markov ...
02/26/2019

Information Gathering in Decentralized POMDPs by Policy Graph Improvement

Decentralized policies for information gathering are required when multi...
08/06/2019

Online Planning for Decentralized Stochastic Control with Partial History Sharing

In decentralized stochastic control, standard approaches for sequential ...
01/16/2014

An Investigation into Mathematical Programming for Finite Horizon Decentralized POMDPs

Decentralized planning in uncertain environments is a complex task gener...
10/31/2011

Communication-Based Decomposition Mechanisms for Decentralized MDPs

Multi-agent planning in stochastic environments can be framed formally a...
02/16/2021

Multi-Stage Transmission Line Flow Control Using Centralized and Decentralized Reinforcement Learning Agents

Planning future operational scenarios of bulk power systems that meet se...