Integrating independent and centralized multi-agent reinforcement learning for traffic signal network optimization

09/23/2019
by   Zhi Zhang, et al.
2

Traffic congestion in metropolitan areas is a world-wide problem that can be ameliorated by traffic lights that respond dynamically to real-time conditions. Recent studies applying deep reinforcement learning (RL) to optimize single traffic lights have shown significant improvement over conventional control. However, optimization of global traffic condition over a large road network fundamentally is a cooperative multi-agent control problem, for which single-agent RL is not suitable due to environment non-stationarity and infeasibility of optimizing over an exponential joint-action space. Motivated by these challenges, we propose QCOMBO, a simple yet effective multi-agent reinforcement learning (MARL) algorithm that combines the advantages of independent and centralized learning. We ensure scalability by selecting actions from individually optimized utility functions, which are shaped to maximize global performance via a novel consistency regularization loss between individual utility and a global action-value function. Experiments on diverse road topologies and traffic flow conditions in the SUMO traffic simulator show competitive performance of QCOMBO versus recent state-of-the-art MARL algorithms. We further show that policies trained on small sub-networks can effectively generalize to larger networks under different traffic flow conditions, providing empirical evidence for the suitability of MARL for intelligent traffic control.

READ FULL TEXT
research
06/05/2023

A Novel Multi-Agent Deep RL Approach for Traffic Signal Control

As travel demand increases and urban traffic condition becomes more comp...
research
08/27/2018

MARL-FWC: Optimal Coordination of Freeway Traffic Control Measures

The objective of this article is to optimize the overall traffic flow on...
research
10/11/2021

Scalable Traffic Signal Controls using Fog-Cloud Based Multiagent Reinforcement Learning

Optimizing traffic signal control (TSC) at intersections continues to po...
research
02/06/2023

Network-Aided Intelligent Traffic Steering in 6G ORAN: A Multi-Layer Optimization Framework

To enable an intelligent, programmable and multi-vendor radio access net...
research
05/19/2020

Batch-Augmented Multi-Agent Reinforcement Learning for Efficient Traffic Signal Optimization

The goal of this work is to provide a viable solution based on reinforce...
research
02/28/2017

Analysing Congestion Problems in Multi-agent Reinforcement Learning

Congestion problems are omnipresent in today's complex networks and repr...
research
11/18/2020

Adaptive Contention Window Design using Deep Q-learning

We study the problem of adaptive contention window (CW) design for rando...

Please sign up or login with your details

Forgot password? Click here to reset