Perimeter Control with Heterogeneous Cordon Signal Behaviors: A Semi-Model Dependent Reinforcement Learning Approach

by   Jiajie Yu, et al.

Perimeter Control (PC) strategies have been proposed to address urban road network control in oversaturated situations by monitoring transfer flows of the Protected Network (PN). The uniform metering rate for cordon signals in existing studies ignores the variety of local traffic states at the intersection level, which may cause severe local traffic congestion and ruin the network stability. This paper introduces a semi-model dependent Multi-Agent Reinforcement Learning (MARL) framework to conduct PC with heterogeneous cordon signal behaviors. The proposed strategy integrates the MARL-based signal control method with centralized feedback PC policy and is applied to cordon signals of the PN. It operates as a two-stage system, with the feedback PC strategy detecting the overall traffic state within the PN and then distributing local instructions to cordon signals controlled by agents in the MARL framework. Each cordon signal acts independently and differently, creating a slack and distributed PC for the PN. The combination of the model-free and model-based methods is achieved by reconstructing the action-value function of the local agents with PC feedback reward without violating the integrity of the local signal control policy learned from the RL training process. Through numerical tests with different demand patterns in a microscopic traffic environment, the proposed PC strategy (a) is shown robustness, scalability, and transferability, (b) outperforms state-of-the-art model-based PC strategies in increasing network throughput, reducing cordon queue and carbon emission.


page 3

page 6

page 11


Lyapunov Function Consistent Adaptive Network Signal Control with Back Pressure and Reinforcement Learning

This research studies the network traffic signal control problem. It use...

A Deep Reinforcement Learning Approach for Traffic Signal Control Optimization

Inefficient traffic signal control methods may cause numerous problems, ...

A self-organizing system for urban traffic control based on predictive interval microscopic model

This paper introduces a self-organizing traffic signal system for an urb...

A Policy Iteration Approach for Flock Motion Control

The flocking motion control is concerned with managing the possible conf...

IG-RL: Inductive Graph Reinforcement Learning for Massive-Scale Traffic Signal Control

Scaling adaptive traffic-signal control involves dealing with combinator...

Cooperative Multi-Objective Reinforcement Learning for Traffic Signal Control and Carbon Emission Reduction

Existing traffic signal control systems rely on oversimplified rule-base...

Quantifying the Impact of Non-Stationarity in Reinforcement Learning-Based Traffic Signal Control

In reinforcement learning (RL), dealing with non-stationarity is a chall...

Please sign up or login with your details

Forgot password? Click here to reset