Large-scale traffic signal control using machine learning: some traffic flow considerations

08/07/2019
by   Jorge A. Laval, et al.
0

This paper uses supervised learning, random search and deep reinforcement learning (DRL) methods to control large signalized intersection networks. The traffic model is Cellular Automaton rule 184, which has been shown to be a parameter-free representation of traffic flow, and is the most efficient implementation of the Kinematic Wave model with triangular fundamental diagram. We are interested in the steady-state performance of the system, both spatially and temporally: we consider a homogeneous grid network inscribed on a torus, which makes the network boundary-free, and drivers choose random routes. As a benchmark we use the longest-queue-first (LQF) greedy algorithm. We find that: (i) a policy trained with supervised learning with only two examples outperforms LQF, (ii) random search is able to generate near-optimal policies, (iii) the prevailing average network occupancy during training is the major determinant of the effectiveness of DRL policies. When trained under free-flow conditions one obtains DRL policies that are optimal for all traffic conditions, but this performance deteriorates as the occupancy during training increases. For occupancies > 75 poorly for all traffic conditions, which means that DRL methods cannot learn under highly congested conditions. We conjecture that DRL's inability to learn under congestion might be explained by a property of urban networks found here, whereby even a very bad policy produces an intersection throughput higher than downstream capacity. This means that the actual throughput tends to be independent of the policy. Our findings imply that it is advisable for current DRL methods in the literature to discard any congested data when training, and that doing this will improve their performance under all traffic conditions.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/21/2020

Congested Urban Networks Tend to Be Insensitive to Signal Settings: Implications for Learning-Based Control

This paper highlights several properties of large urban networks that ca...
research
07/21/2021

A Deep Reinforcement Learning Approach for Fair Traffic Signal Control

Traffic signal control is one of the most effective methods of traffic m...
research
03/04/2023

Demonstration-guided Deep Reinforcement Learning for Coordinated Ramp Metering and Perimeter Control in Large Scale Networks

Effective traffic control methods have great potential in alleviating ne...
research
06/24/2022

Dynamic network congestion pricing based on deep reinforcement learning

Traffic congestion is a serious problem in urban areas. Dynamic congesti...
research
08/21/2022

Development of a CAV-based Intersection Control System and Corridor Level Impact Assessment

This paper presents a signal-free intersection control system for CAVs b...
research
06/20/2023

Neural Inventory Control in Networks via Hindsight Differentiable Policy Optimization

Inventory management offers unique opportunities for reliably evaluating...

Please sign up or login with your details

Forgot password? Click here to reset