Multi-Agent Deep Reinforcement Learning for Large-scale Traffic Signal Control

03/11/2019
by   Tianshu Chu, et al.
22

Reinforcement learning (RL) is a promising data-driven approach for adaptive traffic signal control (ATSC) in complex urban traffic networks, and deep neural networks further enhance its learning power. However, centralized RL is infeasible for large-scale ATSC due to the extremely high dimension of the joint action space. Multi-agent RL (MARL) overcomes the scalability issue by distributing the global control to each local RL agent, but it introduces new challenges: now the environment becomes partially observable from the viewpoint of each local agent due to limited communication among agents. Most existing studies in MARL focus on designing efficient communication and coordination among traditional Q-learning agents. This paper presents, for the first time, a fully scalable and decentralized MARL algorithm for the state-of-the-art deep RL agent: advantage actor critic (A2C), within the context of ATSC. In particular, two methods are proposed to stabilize the learning procedure, by improving the observability and reducing the learning difficulty of each local agent. The proposed multi-agent A2C is compared against independent A2C and independent Q-learning algorithms, in both a large synthetic traffic grid and a large real-world traffic network of Monaco city, under simulated peak-hour traffic dynamics. Results demonstrate its optimality, robustness, and sample efficiency over other state-of-the-art decentralized MARL algorithms.

READ FULL TEXT

page 1

page 7

page 8

page 10

research
06/05/2023

A Novel Multi-Agent Deep RL Approach for Traffic Signal Control

As travel demand increases and urban traffic condition becomes more comp...
research
07/17/2023

Basal-Bolus Advisor for Type 1 Diabetes (T1D) Patients Using Multi-Agent Reinforcement Learning (RL) Methodology

This paper presents a novel multi-agent reinforcement learning (RL) appr...
research
02/18/2018

Efficient Large-Scale Fleet Management via Multi-Agent Deep Reinforcement Learning

Large-scale online ride-sharing platforms have substantially transformed...
research
11/29/2022

Multi-Agent Reinforcement Learning for Microprocessor Design Space Exploration

Microprocessor architects are increasingly resorting to domain-specific ...
research
01/02/2023

Large-Scale Traffic Signal Control by a Nash Deep Q-network Approach

Reinforcement Learning (RL) is currently one of the most commonly used t...
research
06/16/2023

Cooperative Multi-Objective Reinforcement Learning for Traffic Signal Control and Carbon Emission Reduction

Existing traffic signal control systems rely on oversimplified rule-base...
research
08/05/2020

Area-wide traffic signal control based on a deep graph Q-Network (DGQN) trained in an asynchronous manner

Reinforcement learning (RL) algorithms have been widely applied in traff...

Please sign up or login with your details

Forgot password? Click here to reset