Towards a Very Large Scale Traffic Simulator for Multi-Agent Reinforcement Learning Testbeds

by   Zijian Hu, et al.

Smart traffic control and management become an emerging application for Deep Reinforcement Learning (DRL) to solve traffic congestion problems in urban networks. Different traffic control and management policies can be tested on the traffic simulation. Current DRL-based studies are mainly supported by the microscopic simulation software (e.g., SUMO), while it is not suitable for city-wide control due to the computational burden and gridlock effect. To the best of our knowledge, there is a lack of studies on the large-scale traffic simulator for DRL testbeds, which could further hinder the development of DRL. In view of this, we propose a meso-macro traffic simulator for very large-scale DRL scenarios. The proposed simulator integrates mesoscopic and macroscopic traffic simulation models to improve efficiency and eliminate gridlocks. The mesoscopic link model simulates flow dynamics on roads, and the macroscopic Bathtub model depicts vehicle movement in regions. Moreover, both types of models can be hybridized to accommodate various DRL tasks. This creates portals for mixed transportation applications under different contexts. The result shows that the developed simulator only takes 46 seconds to finish a 24-hour simulation in a very large city with 2.2 million vehicles, which is much faster than SUMO. Additionally, we develop a graphic interface for users to visualize the simulation results in a web explorer. In the future, the developed meso-macro traffic simulator could serve as a new environment for very large-scale DRL problems.


page 4

page 6


Demonstration-guided Deep Reinforcement Learning for Coordinated Ramp Metering and Perimeter Control in Large Scale Networks

Effective traffic control methods have great potential in alleviating ne...

CityFlow: A Multi-Agent Reinforcement Learning Environment for Large Scale City Traffic Scenario

Traffic signal control is an emerging application scenario for reinforce...

QarSUMO: A Parallel, Congestion-optimized Traffic Simulator

Traffic simulators are important tools for tasks such as urban planning ...

TransWorldNG: Traffic Simulation via Foundation Model

Traffic simulation is a crucial tool for transportation decision-making ...

Learning to Route via Theory-Guided Residual Network

The heavy traffic and related issues have always been concerns for moder...

LibSignal: An Open Library for Traffic Signal Control

This paper introduces a library for cross-simulator comparison of reinfo...

Optimal Smoothing Distribution Exploration for Backdoor Neutralization in Deep Learning-based Traffic Systems

Deep Reinforcement Learning (DRL) enhances the efficiency of Autonomous ...

Please sign up or login with your details

Forgot password? Click here to reset