DeepAI AI Chat
Log In Sign Up

Scaling Multi-Agent Reinforcement Learning with Selective Parameter Sharing

by   Filippos Christianos, et al.

Sharing parameters in multi-agent deep reinforcement learning has played an essential role in allowing algorithms to scale to a large number of agents. Parameter sharing between agents significantly decreases the number of trainable parameters, shortening training times to tractable levels, and has been linked to more efficient learning. However, having all agents share the same parameters can also have a detrimental effect on learning. We demonstrate the impact of parameter sharing methods on training speed and converged returns, establishing that when applied indiscriminately, their effectiveness is highly dependent on the environment. Therefore, we propose a novel method to automatically identify agents which may benefit from sharing parameters by partitioning them based on their abilities and goals. Our approach combines the increased sample efficiency of parameter sharing with the representational capacity of multiple independent networks to reduce training time and increase final returns.


page 5

page 7


Parameter Sharing with Network Pruning for Scalable Multi-Agent Deep Reinforcement Learning

Handling the problem of scalability is one of the essential issues for m...

Parameter Sharing is Surprisingly Useful for Multi-Agent Deep Reinforcement Learning

"Nonstationarity" is a fundamental problem in cooperative multi-agent re...

KnowSR: Knowledge Sharing among Homogeneous Agents in Multi-agent Reinforcement Learning

Recently, deep reinforcement learning (RL) algorithms have made great pr...

Discriminative Experience Replay for Efficient Multi-agent Reinforcement Learning

In cooperative multi-agent tasks, parameter sharing among agents is a co...

Experience Sharing Between Cooperative Reinforcement Learning Agents

The idea of experience sharing between cooperative agents naturally emer...

Scalable Evaluation of Multi-Agent Reinforcement Learning with Melting Pot

Existing evaluation suites for multi-agent reinforcement learning (MARL)...

Parameter Sharing Reinforcement Learning Architecture for Multi Agent Driving Behaviors

Multi-agent learning provides a potential framework for learning and sim...