Reinforcement Learning for Heterogeneous Teams with PALO Bounds

05/23/2018
by   Roi Ceren, et al.
0

We introduce reinforcement learning for heterogeneous teams in which rewards for an agent are additively factored into local costs, stimuli unique to each agent, and global rewards, those shared by all agents in the domain. Motivating domains include coordination of varied robotic platforms, which incur different costs for the same action, but share an overall goal. We present two templates for learning in this setting with factored rewards: a generalization of Perkins' Monte Carlo exploring starts for POMDPs to canonical MPOMDPs, with a single policy mapping joint observations of all agents to joint actions (MCES-MP); and another with each agent individually mapping joint observations to their own action (MCES-FMP). We use probably approximately local optimal (PALO) bounds to analyze sample complexity, instantiating these templates to PALO learning. We promote sample efficiency by including a policy space pruning technique, and evaluate the approaches on three domains of heterogeneous agents demonstrating that MCES-FMP yields improved policies in less samples compared to MCES-MP and a previous benchmark.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/13/2018

Learning to Communicate: A Machine Learning Framework for Heterogeneous Multi-Agent Robotic Systems

We present a machine learning framework for multi-agent systems to learn...
research
05/28/2019

Coordinated Exploration via Intrinsic Rewards for Multi-Agent Reinforcement Learning

Sparse rewards are one of the most important challenges in reinforcement...
research
09/07/2022

On the Near-Optimality of Local Policies in Large Cooperative Multi-Agent Reinforcement Learning

We show that in a cooperative N-agent network, one can design locally ex...
research
10/22/2021

Convergence Rates of Average-Reward Multi-agent Reinforcement Learning via Randomized Linear Programming

In tabular multi-agent reinforcement learning with average-cost criterio...
research
06/18/2019

Evolutionary Reinforcement Learning for Sample-Efficient Multiagent Coordination

A key challenge for Multiagent RL (Reinforcement Learning) is the design...
research
03/07/2019

Learning Hierarchical Teaching in Cooperative Multiagent Reinforcement Learning

Heterogeneous knowledge naturally arises among different agents in coope...
research
11/21/2017

Transferring Agent Behaviors from Videos via Motion GANs

A major bottleneck for developing general reinforcement learning agents ...

Please sign up or login with your details

Forgot password? Click here to reset