GeneraLight: Improving Environment Generalization of Traffic Signal Control via Meta Reinforcement Learning

09/17/2020
by   Chang Liu, et al.
9

The heavy traffic congestion problem has always been a concern for modern cities. To alleviate traffic congestion, researchers use reinforcement learning (RL) to develop better traffic signal control (TSC) algorithms in recent years. However, most RL models are trained and tested in the same traffic flow environment, which results in a serious overfitting problem. Since the traffic flow environment in the real world keeps varying, these models can hardly be applied due to the lack of generalization ability. Besides, the limited number of accessible traffic flow data brings extra difficulty in testing the generalization ability of the models. In this paper, we design a novel traffic flow generator based on Wasserstein generative adversarial network to generate sufficient diverse and quality traffic flows and use them to build proper training and testing environments. Then we propose a meta-RL TSC framework GeneraLight to improve the generalization ability of TSC models. GeneraLight boosts the generalization performance by combining the idea of flow clustering and model-agnostic meta-learning. We conduct extensive experiments on multiple real-world datasets to show the superior performance of GeneraLight on generalizing to different traffic flows.

READ FULL TEXT

page 1

page 2

page 3

page 4

11/15/2021

ModelLight: Model-Based Meta-Reinforcement Learning for Traffic Signal Control

Traffic signal control is of critical importance for the effective use o...
09/15/2021

Back to Basics: Deep Reinforcement Learning in Traffic Signal Control

In this paper we revisit some of the fundamental premises for a reinforc...
10/12/2020

AttendLight: Universal Attention-Based Reinforcement Learning Model for Traffic Signal Control

We propose AttendLight, an end-to-end Reinforcement Learning (RL) algori...
06/30/2021

Exploring Context Modeling Techniques on the Spatiotemporal Crowd Flow Prediction

In the big data and AI era, context is widely exploited as extra informa...
05/12/2021

An Empirical Experiment on Deep Learning Models for Predicting Traffic Data

To tackle ever-increasing city traffic congestion problems, researchers ...
05/18/2021

Learning to Route via Theory-Guided Residual Network

The heavy traffic and related issues have always been concerns for moder...