Learning to Simulate Self-Driven Particles System with Coordinated Policy Optimization

10/26/2021
by   Zhenghao Peng, et al.
0

Self-Driven Particles (SDP) describe a category of multi-agent systems common in everyday life, such as flocking birds and traffic flows. In a SDP system, each agent pursues its own goal and constantly changes its cooperative or competitive behaviors with its nearby agents. Manually designing the controllers for such SDP system is time-consuming, while the resulting emergent behaviors are often not realistic nor generalizable. Thus the realistic simulation of SDP systems remains challenging. Reinforcement learning provides an appealing alternative for automating the development of the controller for SDP. However, previous multi-agent reinforcement learning (MARL) methods define the agents to be teammates or enemies before hand, which fail to capture the essence of SDP where the role of each agent varies to be cooperative or competitive even within one episode. To simulate SDP with MARL, a key challenge is to coordinate agents' behaviors while still maximizing individual objectives. Taking traffic simulation as the testing bed, in this work we develop a novel MARL method called Coordinated Policy Optimization (CoPO), which incorporates social psychology principle to learn neural controller for SDP. Experiments show that the proposed method can achieve superior performance compared to MARL baselines in various metrics. Noticeably the trained vehicles exhibit complex and diverse social behaviors that improve performance and safety of the population as a whole. Demo video and source code are available at: https://decisionforce.github.io/CoPO/

READ FULL TEXT

page 2

page 21

research
01/17/2021

TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviors

Simulation has the potential to massively scale evaluation of self-drivi...
research
09/02/2021

MACRPO: Multi-Agent Cooperative Recurrent Policy Optimization

This work considers the problem of learning cooperative policies in mult...
research
03/28/2022

UNMAS: Multi-Agent Reinforcement Learning for Unshaped Cooperative Scenarios

Multi-agent reinforcement learning methods such as VDN, QMIX, and QTRAN ...
research
04/20/2023

Interpretability for Conditional Coordinated Behavior in Multi-Agent Reinforcement Learning

We propose a model-free reinforcement learning architecture, called dist...
research
11/17/2018

Parameter Sharing Reinforcement Learning Architecture for Multi Agent Driving Behaviors

Multi-agent learning provides a potential framework for learning and sim...
research
02/10/2023

Learning cooperative behaviours in adversarial multi-agent systems

This work extends an existing virtual multi-agent platform called RoboSu...
research
03/16/2022

PMIC: Improving Multi-Agent Reinforcement Learning with Progressive Mutual Information Collaboration

Learning to collaborate is critical in Multi-Agent Reinforcement Learnin...

Please sign up or login with your details

Forgot password? Click here to reset