Cooperative Reinforcement Learning on Traffic Signal Control

05/23/2022
by   Chi-Chun Chao, et al.
0

Traffic signal control is a challenging real-world problem aiming to minimize overall travel time by coordinating vehicle movements at road intersections. Existing traffic signal control systems in use still rely heavily on oversimplified information and rule-based methods. Specifically, the periodicity of green/red light alternations can be considered as a prior for better planning of each agent in policy optimization. To better learn such adaptive and predictive priors, traditional RL-based methods can only return a fixed length from predefined action pool with only local agents. If there is no cooperation between these agents, some agents often make conflicts to other agents and thus decrease the whole throughput. This paper proposes a cooperative, multi-objective architecture with age-decaying weights to better estimate multiple reward terms for traffic signal control optimization, which termed COoperative Multi-Objective Multi-Agent Deep Deterministic Policy Gradient (COMMA-DDPG). Two types of agents running to maximize rewards of different goals - one for local traffic optimization at each intersection and the other for global traffic waiting time optimization. The global agent is used to guide the local agents as a means for aiding faster learning but not used in the inference phase. We also provide an analysis of solution existence together with convergence proof for the proposed RL optimization. Evaluation is performed using real-world traffic data collected using traffic cameras from an Asian country. Our method can effectively reduce the total delayed time by 60%. Results demonstrate its superiority when compared to SoTA methods.

READ FULL TEXT

page 1

page 4

research
06/16/2023

Cooperative Multi-Objective Reinforcement Learning for Traffic Signal Control and Carbon Emission Reduction

Existing traffic signal control systems rely on oversimplified rule-base...
research
04/17/2019

A Survey on Traffic Signal Control Methods

Traffic signal control is an important and challenging real-world proble...
research
04/21/2023

Reinforcement Learning Approaches for Traffic Signal Control under Missing Data

The emergence of reinforcement learning (RL) methods in traffic signal c...
research
11/22/2020

Multi-Agent Reinforcement Learning for Dynamic Routing Games: A Unified Paradigm

This paper aims to develop a unified paradigm that models one's learning...
research
11/20/2022

SafeLight: A Reinforcement Learning Method toward Collision-free Traffic Signal Control

Traffic signal control is safety-critical for our daily life. Roughly on...
research
03/27/2017

Deep Deterministic Policy Gradient for Urban Traffic Light Control

Traffic light timing optimization is still an active line of research de...
research
09/30/2019

Tensor-based Cooperative Control for Large Scale Multi-intersection Traffic Signal Using Deep Reinforcement Learning and Imitation Learning

Traffic signal control has long been considered as a critical topic in i...

Please sign up or login with your details

Forgot password? Click here to reset