Area-wide traffic signal control based on a deep graph Q-Network (DGQN) trained in an asynchronous manner

08/05/2020
by   Gyeongjun Kim, et al.
0

Reinforcement learning (RL) algorithms have been widely applied in traffic signal studies. There are, however, several problems in jointly controlling traffic lights for a large transportation network. First, the action space exponentially explodes as the number of intersections to be jointly controlled increases. Although a multi-agent RL algorithm has been used to solve the curse of dimensionality, this neither guaranteed a global optimum, nor could it break the ties between joint actions. The problem was circumvented by revising the output structure of a deep Q-network (DQN) within the framework of a single-agent RL algorithm. Second, when mapping traffic states into an action value, it is difficult to consider spatio-temporal correlations over a large transportation network. A deep graph Q-network (DGQN) was devised to efficiently accommodate spatio-temporal dependencies on a large scale. Finally, training a RL model to jointly control traffic lights in a large transportation network requires much time to converge. An asynchronous update methodology was devised for a DGQN to quickly reach an optimal policy. Using these three remedies, a DGQN succeeded in jointly controlling the traffic lights in a large transportation network in Seoul. This approach outperformed other state-of-the-art RL algorithms as well as an actual fixed-signal operation.

READ FULL TEXT

page 21

page 29

research
06/05/2023

A Novel Multi-Agent Deep RL Approach for Traffic Signal Control

As travel demand increases and urban traffic condition becomes more comp...
research
03/11/2019

Multi-Agent Deep Reinforcement Learning for Large-scale Traffic Signal Control

Reinforcement learning (RL) is a promising data-driven approach for adap...
research
11/11/2018

Optimizing Taxi Carpool Policies via Reinforcement Learning and Spatio-Temporal Mining

In this paper, we develop a reinforcement learning (RL) based system to ...
research
01/02/2023

Large-Scale Traffic Signal Control by a Nash Deep Q-network Approach

Reinforcement Learning (RL) is currently one of the most commonly used t...
research
10/11/2021

Scalable Traffic Signal Controls using Fog-Cloud Based Multiagent Reinforcement Learning

Optimizing traffic signal control (TSC) at intersections continues to po...
research
04/07/2022

DynLight: Realize dynamic phase duration with multi-level traffic signal control

Adopting reinforcement learning (RL) for traffic signal control is incre...
research
02/27/2020

Learning Scalable Multi-Agent Coordination by Spatial Differential for Traffic Signal Control

The intelligent control of the traffic signal is critical to the optimiz...

Please sign up or login with your details

Forgot password? Click here to reset