Structured Diversification Emergence via Reinforced Organization Control and Hierarchical Consensus Learning

02/09/2021
by   Wenhao Li, et al.
0

When solving a complex task, humans will spontaneously form teams and to complete different parts of the whole task, respectively. Meanwhile, the cooperation between teammates will improve efficiency. However, for current cooperative MARL methods, the cooperation team is constructed through either heuristics or end-to-end blackbox optimization. In order to improve the efficiency of cooperation and exploration, we propose a structured diversification emergence MARL framework named Rochico based on reinforced organization control and hierarchical consensus learning. Rochico first learns an adaptive grouping policy through the organization control module, which is established by independent multi-agent reinforcement learning. Further, the hierarchical consensus module based on the hierarchical intentions with consensus constraint is introduced after team formation. Simultaneously, utilizing the hierarchical consensus module and a self-supervised intrinsic reward enhanced decision module, the proposed cooperative MARL algorithm Rochico can output the final diversified multi-agent cooperative policy. All three modules are organically combined to promote the structured diversification emergence. Comparative experiments on four large-scale cooperation tasks show that Rochico is significantly better than the current SOTA algorithms in terms of exploration efficiency and cooperation strength.

READ FULL TEXT
research
01/05/2023

Self-Motivated Multi-Agent Exploration

In cooperative multi-agent reinforcement learning (CMARL), it is critica...
research
08/21/2020

Learning to Collaborate in Multi-Module Recommendation via Multi-Agent Reinforcement Learning without Communication

With the rise of online e-commerce platforms, more and more customers pr...
research
06/10/2020

The Emergence of Individuality in Multi-Agent Reinforcement Learning

Individuality is essential in human society, which induces the division ...
research
02/11/2020

Learning Structured Communication for Multi-agent Reinforcement Learning

This work explores the large-scale multi-agent communication mechanism u...
research
07/28/2023

Learning to Collaborate by Grouping: a Consensus-oriented Strategy for Multi-agent Reinforcement Learning

Multi-agent systems require effective coordination between groups and in...
research
08/05/2022

A Cooperation Graph Approach for Multiagent Sparse Reward Reinforcement Learning

Multiagent reinforcement learning (MARL) can solve complex cooperative t...
research
11/17/2018

Emergence of linguistic conventions in multi-agent reinforcement learning

Recently, emergence of signaling conventions, among which language is a ...

Please sign up or login with your details

Forgot password? Click here to reset