DeepAI AI Chat
Log In Sign Up

Evolutionary Reinforcement Learning for Sample-Efficient Multiagent Coordination

by   Shauharda Khadka, et al.
Oregon State University

A key challenge for Multiagent RL (Reinforcement Learning) is the design of agent-specific, local rewards that are aligned with sparse global objectives. In this paper, we introduce MERL (Multiagent Evolutionary RL), a hybrid algorithm that does not require an explicit alignment between local and global objectives. MERL uses fast, policy-gradient based learning for each agent by utilizing their dense local rewards. Concurrently, an evolutionary algorithm is used to recruit agents into a team by directly optimizing the sparser global objective. We explore problems that require coupling (a minimum number of agents required to coordinate for success), where the degree of coupling is not known to the agents. We demonstrate that MERL's integrated approach is more sample-efficient and retains performance better with increasing coupling orders compared to MADDPG, the state-of-the-art policy-gradient algorithm for multiagent coordination.


page 3

page 8


Supplementing Gradient-Based Reinforcement Learning with Simple Evolutionary Ideas

We present a simple, sample-efficient algorithm for introducing large bu...

Policy Gradient for Reinforcement Learning with General Utilities

In Reinforcement Learning (RL), the goal of agents is to discover an opt...

Joint Optimization of Multi-Objective Reinforcement Learning with Policy Gradient Based Algorithm

Many engineering problems have multiple objectives, and the overall aim ...

Cooperative Heterogeneous Deep Reinforcement Learning

Numerous deep reinforcement learning agents have been proposed, and each...

Reinforcement Learning for Heterogeneous Teams with PALO Bounds

We introduce reinforcement learning for heterogeneous teams in which rew...

Quantile-Based Policy Optimization for Reinforcement Learning

Classical reinforcement learning (RL) aims to optimize the expected cumu...

Evolutionary RL for Container Loading

Loading the containers on the ship from a yard, is an impor- tant part o...