Towards Learning Multi-agent Negotiations via Self-Play

by   Yichuan Charlie Tang, et al.

Making sophisticated, robust, and safe sequential decisions is at the heart of intelligent systems. This is especially critical for planning in complex multi-agent environments, where agents need to anticipate other agents' intentions and possible future actions. Traditional methods formulate the problem as a Markov Decision Process, but the solutions often rely on various assumptions and become brittle when presented with corner cases. In contrast, deep reinforcement learning (Deep RL) has been very effective at finding policies by simultaneously exploring, interacting, and learning from environments. Leveraging the powerful Deep RL paradigm, we demonstrate that an iterative procedure of self-play can create progressively more diverse environments, leading to the learning of sophisticated and robust multi-agent policies. We demonstrate this in a challenging multi-agent simulation of merging traffic, where agents must interact and negotiate with others in order to successfully merge on or off the road. While the environment starts off simple, we increase its complexity by iteratively adding an increasingly diverse set of agents to the agent "zoo" as training progresses. Qualitatively, we find that through self-play, our policies automatically learn interesting behaviors such as defensive driving, overtaking, yielding, and the use of signal lights to communicate intentions to other agents. In addition, quantitatively, we show a dramatic improvement of the success rate of merging maneuvers from 63


page 1

page 3

page 6

page 8

page 9


Deep Reinforcement Learning for Multi-Agent Systems: A Review of Challenges, Solutions and Applications

Reinforcement learning (RL) algorithms have been around for decades and ...

RPM: Generalizable Behaviors for Multi-Agent Reinforcement Learning

Despite the recent advancement in multi-agent reinforcement learning (MA...

Multi-Agent Connected Autonomous Driving using Deep Reinforcement Learning

The capability to learn and adapt to changes in the driving environment ...

Model-based Dynamic Shielding for Safe and Efficient Multi-Agent Reinforcement Learning

Multi-Agent Reinforcement Learning (MARL) discovers policies that maximi...

Interactive POMDP Lite: Towards Practical Planning to Predict and Exploit Intentions for Interacting with Self-Interested Agents

A key challenge in non-cooperative multi-agent systems is that of develo...

FormulaZero: Distributionally Robust Online Adaptation via Offline Population Synthesis

Balancing performance and safety is crucial to deploying autonomous vehi...

Adversarially Guided Self-Play for Adopting Social Conventions

Robotic agents must adopt existing social conventions in order to be eff...

Please sign up or login with your details

Forgot password? Click here to reset