Policy Synthesis for Factored MDPs with Graph Temporal Logic Specifications

by   Murat Cubuktepe, et al.

We study the synthesis of policies for multi-agent systems to implement spatial-temporal tasks. We formalize the problem as a factored Markov decision process subject to so-called graph temporal logic specifications. The transition function and the spatial-temporal task of each agent depend on the agent itself and its neighboring agents. The structure in the model and the specifications enable to develop a distributed algorithm that, given a factored Markov decision process and a graph temporal logic formula, decomposes the synthesis problem into a set of smaller synthesis problems, one for each agent. We prove that the algorithm runs in time linear in the total number of agents. The size of the synthesis problem for each agent is exponential only in the number of neighboring agents, which is typically much smaller than the number of agents. We demonstrate the algorithm in case studies on disease control and urban security. The numerical examples show that the algorithm can scale to hundreds of agents.


Distributed Policy Synthesis of Multi-Agent Systems With Graph Temporal Logic Specifications

We study the distributed synthesis of policies for multi-agent systems t...

Strategy Synthesis for Global Window PCTL

Given a Markov decision process (MDP) M and a formula Φ, the strategy sy...

Distributed Locally Non-interfering Connectivity via Linear Temporal Logic

In this paper, we consider networks of static sensors with integrated se...

Polynomial-Time Algorithms for Multi-Agent Minimal-Capacity Planning

We study the problem of minimizing the resource capacity of autonomous a...

Interpretable Apprenticeship Learning with Temporal Logic Specifications

Recent work has addressed using formulas in linear temporal logic (LTL) ...

Large-scale, Dynamic and Distributed Coalition Formation with Spatial and Temporal Constraints

The Coalition Formation with Spatial and Temporal constraints Problem (C...

Probabilistic Control of Heterogeneous Swarms Subject to Graph Temporal Logic Specifications: A Decentralized and Scalable Approach

We develop a probabilistic control algorithm, , for swarms of agents wit...