Multi-Agent Reinforcement Learning for Dynamic Routing Games: A Unified Paradigm

11/22/2020
by   Zhenyu Shou, et al.
0

This paper aims to develop a unified paradigm that models one's learning behavior and the system's equilibrating processes in a routing game among atomic selfish agents. Such a paradigm can assist policymakers in devising optimal operational and planning countermeasures under both normal and abnormal circumstances. To this end, a multi-agent reinforcement learning (MARL) paradigm is proposed in which each agent learns and updates her own en-route path choice policy while interacting with others on transportation networks. This paradigm is shown to generalize the classical notion of dynamic user equilibrium (DUE) to model-free and data-driven scenarios. We also illustrate that the equilibrium outcomes computed from our developed MARL paradigm coincide with DUE and dynamic system optimal (DSO), respectively, when rewards are set differently. In addition, with the goal to optimize some systematic objective (e.g., overall traffic condition) of city planners, we formulate a bilevel optimization problem with the upper level as city planners and the lower level as a multi-agent system where each rational and selfish traveler aims to minimize her travel cost. We demonstrate the effect of two administrative measures, namely tolling and signal control, on the behavior of travelers and show that the systematic objective of city planners can be optimized by a proper control. The results show that on the Braess network, the optimal toll charge on the central link is greater or equal to 25, with which the average travel time of selfish agents is minimized and the emergence of Braess paradox could be avoided. In a large-sized real-world road network with 69 nodes and 166 links, the optimal offset for signal control on Broadway is derived as 4 seconds, with which the average travel time of all controllable agents is minimized.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/27/2022

EMVLight: a Multi-agent Reinforcement Learning Framework for an Emergency Vehicle Decentralized Routing and Traffic Signal Control System

Emergency vehicles (EMVs) play a crucial role in responding to time-crit...
research
10/30/2021

A Decentralized Reinforcement Learning Framework for Efficient Passage of Emergency Vehicles

Emergency vehicles (EMVs) play a critical role in a city's response to t...
research
05/23/2022

Cooperative Reinforcement Learning on Traffic Signal Control

Traffic signal control is a challenging real-world problem aiming to min...
research
05/02/2021

Reducing Bus Bunching with Asynchronous Multi-Agent Reinforcement Learning

The bus system is a critical component of sustainable urban transportati...
research
11/09/2019

Empirical validation of network learning with taxi GPS data from Wuhan, China

In prior research, a statistically cheap method was developed to monitor...
research
02/17/2020

Reward Design for Driver Repositioning Using Multi-Agent Reinforcement Learning

A large portion of the passenger requests is reportedly unserviced, part...
research
10/17/2021

Coordinated Multi-Agent Pathfinding for Drones and Trucks over Road Networks

We address the problem of routing a team of drones and trucks over large...

Please sign up or login with your details

Forgot password? Click here to reset