DeepAI AI Chat
Log In Sign Up

Quantifying the Impact of Non-Stationarity in Reinforcement Learning-Based Traffic Signal Control

04/09/2020
by   Lucas N. Alegre, et al.
UFRGS
0

In reinforcement learning (RL), dealing with non-stationarity is a challenging issue. However, some domains such as traffic optimization are inherently non-stationary. Causes for and effects of this are manifold. In particular, when dealing with traffic signal controls, addressing non-stationarity is key since traffic conditions change over time and as a function of traffic control decisions taken in other parts of a network. In this paper we analyze the effects that different sources of non-stationarity have in a network of traffic signals, in which each signal is modeled as a learning agent. More precisely, we study both the effects of changing the context in which an agent learns (e.g., a change in flow rates experienced by it), as well as the effects of reducing agent observability of the true environment state. Partial observability may cause distinct states (in which distinct actions are optimal) to be seen as the same by the traffic signal agents. This, in turn, may lead to sub-optimal performance. We show that the lack of suitable sensors to provide a representative observation of the real state seems to affect the performance more drastically than the changes to the underlying traffic patterns.

READ FULL TEXT
05/10/2019

Reinforcement Learning in Non-Stationary Environments

Reinforcement learning (RL) methods learn optimal decisions in the prese...
04/21/2023

Reinforcement Learning Approaches for Traffic Signal Control under Missing Data

The emergence of reinforcement learning (RL) methods in traffic signal c...
03/21/2023

Multi-agent Reinforcement Learning for Regional Signal control in Large-scale Grid Traffic network

Adaptive traffic signal control with Multi-agent Reinforcement Learning(...
03/06/2020

IG-RL: Inductive Graph Reinforcement Learning for Massive-Scale Traffic Signal Control

Scaling adaptive traffic-signal control involves dealing with combinator...
06/04/2014

A self-organizing system for urban traffic control based on predictive interval microscopic model

This paper introduces a self-organizing traffic signal system for an urb...
09/17/2020

GeneraLight: Improving Environment Generalization of Traffic Signal Control via Meta Reinforcement Learning

The heavy traffic congestion problem has always been a concern for moder...
04/22/2021

Independent Reinforcement Learning for Weakly Cooperative Multiagent Traffic Control Problem

The adaptive traffic signal control (ATSC) problem can be modeled as a m...