STMARL: A Spatio-Temporal Multi-Agent Reinforcement Learning Approach for Traffic Light Control

08/28/2019
by   Yanan Wang, et al.
13

The development of intelligent traffic light control systems is essential for smart transportation management. While some efforts have been made to optimize the use of individual traffic lights in an isolated way, related studies have largely ignored the fact that the use of multi-intersection traffic lights is spatially influenced and there is a temporal dependency of historical traffic status for current traffic light control. To that end, in this paper, we propose a novel SpatioTemporal Multi-Agent Reinforcement Learning (STMARL) framework for effectively capturing the spatio-temporal dependency of multiple related traffic lights and control these traffic lights in a coordinating way. Specifically, we first construct the traffic light adjacency graph based on the spatial structure among traffic lights. Then, historical traffic records will be integrated with current traffic status via Recurrent Neural Network structure. Moreover, based on the temporally-dependent traffic information, we design a Graph Neural Network based model to represent relationships among multiple traffic lights, and the decision for each traffic light will be made in a distributed way by the deep Q-learning method. Finally, the experimental results on both synthetic and real-world data have demonstrated the effectiveness of our STMARL framework, which also provides an insightful understanding of the influence mechanism among multi-intersection traffic lights.

READ FULL TEXT

page 3

page 12

research
03/08/2022

Multi-Agent Broad Reinforcement Learning for Intelligent Traffic Light Control

Intelligent Traffic Light Control System (ITLCS) is a typical Multi-Agen...
research
07/14/2023

LightFormer: An End-to-End Model for Intersection Right-of-Way Recognition Using Traffic Light Signals and an Attention Mechanism

For smart vehicles driving through signalised intersections, it is cruci...
research
09/12/2021

DynSTGAT: Dynamic Spatial-Temporal Graph Attention Network for Traffic Signal Control

Adaptive traffic signal control plays a significant role in the construc...
research
07/18/2022

An Enhanced Graph Representation for Machine Learning Based Automatic Intersection Management

The improvement of traffic efficiency at urban intersections receives st...
research
09/30/2019

Tensor-based Cooperative Control for Large Scale Multi-intersection Traffic Signal Using Deep Reinforcement Learning and Imitation Learning

Traffic signal control has long been considered as a critical topic in i...
research
01/24/2021

Multi-intersection Traffic Optimisation: A Benchmark Dataset and a Strong Baseline

The control of traffic signals is fundamental and critical to alleviate ...
research
12/28/2022

Coordination of Drones at Scale: Decentralized Energy-aware Swarm Intelligence for Spatio-temporal Sensing

Smart City applications, such as traffic monitoring and disaster respons...

Please sign up or login with your details

Forgot password? Click here to reset