Multi-Agent Fully Decentralized Off-Policy Learning with Linear Convergence Rates

10/17/2018
by   Lucas Cassano, et al.
0

In this paper we develop a fully decentralized algorithm for policy evaluation with off-policy learning, linear function approximation, and O(n) complexity in both computation and memory requirements. The proposed algorithm is of the variance reduced kind and achieves linear convergence. We consider the case where a collection of agents have distinct and fixed size datasets gathered following different behavior policies (none of which is required to explore the full state space) and they all collaborate to evaluate a common target policy. The network approach allows all agents to converge to the optimal solution even in situations where neither agent can converge on its own without cooperation. We provide simulations to illustrate the effectiveness of the method.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/17/2018

Multi-Agent Fully Decentralized Value Function Learning with Linear Convergence Rates

This work develops a fully decentralized multi-agent algorithm for polic...
research
12/30/2013

Distributed Policy Evaluation Under Multiple Behavior Strategies

We apply diffusion strategies to develop a fully-distributed cooperative...
research
03/03/2021

Inference-Based Deterministic Messaging For Multi-Agent Communication

Communication is essential for coordination among humans and animals. Th...
research
05/29/2018

Learning Under Distributed Features

This work studies the problem of learning under both large data and larg...
research
05/01/2015

Stick-Breaking Policy Learning in Dec-POMDPs

Expectation maximization (EM) has recently been shown to be an efficient...
research
03/19/2020

Decentralized MCTS via Learned Teammate Models

A key difficulty of cooperative decentralized planning lies in making ac...
research
03/15/2012

Rollout Sampling Policy Iteration for Decentralized POMDPs

We present decentralized rollout sampling policy iteration (DecRSPI) - a...

Please sign up or login with your details

Forgot password? Click here to reset