DeepAI AI Chat
Log In Sign Up

Hierarchical Reinforcement Learning with Opponent Modeling for Distributed Multi-agent Cooperation

by   Zhixuan Liang, et al.

Many real-world applications can be formulated as multi-agent cooperation problems, such as network packet routing and coordination of autonomous vehicles. The emergence of deep reinforcement learning (DRL) provides a promising approach for multi-agent cooperation through the interaction of the agents and environments. However, traditional DRL solutions suffer from the high dimensions of multiple agents with continuous action space during policy search. Besides, the dynamicity of agents' policies makes the training non-stationary. To tackle the issues, we propose a hierarchical reinforcement learning approach with high-level decision-making and low-level individual control for efficient policy search. In particular, the cooperation of multiple agents can be learned in high-level discrete action space efficiently. At the same time, the low-level individual control can be reduced to single-agent reinforcement learning. In addition to hierarchical reinforcement learning, we propose an opponent modeling network to model other agents' policies during the learning process. In contrast to end-to-end DRL approaches, our approach reduces the learning complexity by decomposing the overall task into sub-tasks in a hierarchical way. To evaluate the efficiency of our approach, we conduct a real-world case study in the cooperative lane change scenario. Both simulation and real-world experiments show the superiority of our approach in the collision rate and convergence speed.


page 2

page 3

page 4

page 5

page 6

page 8

page 10

page 11


Directionality Reinforcement Learning to Operate Multi-Agent System without Communication

This paper establishes directionality reinforcement learning (DRL) techn...

Hierarchical Reinforcement Learning Framework towards Multi-agent Navigation

This paper proposes a navigation algorithm ori- ented to multi-agent dyn...

Learning Graph-Enhanced Commander-Executor for Multi-Agent Navigation

This paper investigates the multi-agent navigation problem, which requir...

Hierarchical Multi-Agent Reinforcement Learning for Air Combat Maneuvering

The application of artificial intelligence to simulate air-to-air combat...

Scalable Voltage Control using Structure-Driven Hierarchical Deep Reinforcement Learning

This paper presents a novel hierarchical deep reinforcement learning (DR...

A Hierarchical Approach to Population Training for Human-AI Collaboration

A major challenge for deep reinforcement learning (DRL) agents is to col...

Multi-focus Attention Network for Efficient Deep Reinforcement Learning

Deep reinforcement learning (DRL) has shown incredible performance in le...