DeepAI
Log In Sign Up

Compact Mathematical Programs For DEC-MDPs With Structured Agent Interactions

02/14/2012
by   Hala Mostafa, et al.
0

To deal with the prohibitive complexity of calculating policies in Decentralized MDPs, researchers have proposed models that exploit structured agent interactions. Settings where most agent actions are independent except for few actions that affect the transitions and/or rewards of other agents can be modeled using Event-Driven Interactions with Complex Rewards (EDI-CR). Finding the optimal joint policy can be formulated as an optimization problem. However, existing formulations are too verbose and/or lack optimality guarantees. We propose a compact Mixed Integer Linear Program formulation of EDI-CR instances. The key insight is that most action sequences of a group of agents have the same effect on a given agent. This allows us to treat these sequences similarly and use fewer variables. Experiments show that our formulation is more compact and leads to faster solution times and better solutions than existing formulations.

READ FULL TEXT
12/06/2017

Lifting Linear Extension Complexity Bounds to the Mixed-Integer Setting

Mixed-integer mathematical programs are among the most commonly used mod...
01/16/2014

Resource-Driven Mission-Phasing Techniques for Constrained Agents in Stochastic Environments

Because an agents resources dictate what actions it can possibly take, i...
11/29/2015

Solving Transition-Independent Multi-agent MDPs with Sparse Interactions (Extended version)

In cooperative multi-agent sequential decision making under uncertainty,...
01/23/2019

Learning to Collaborate in Markov Decision Processes

We consider a two-agent MDP framework where agents repeatedly solve a ta...
08/09/2022

Branching Pomsets for Choreographies

Choreographic languages describe possible sequences of interactions amon...
02/08/2021

Escaping Stochastic Traps with Aleatoric Mapping Agents

Exploration in environments with sparse rewards is difficult for artific...
09/26/2013

Qualitative Possibilistic Mixed-Observable MDPs

Possibilistic and qualitative POMDPs (pi-POMDPs) are counterparts of POM...