DQN Control Solution for KDD Cup 2021 City Brain Challenge

by   Yitian Chen, et al.
Alibaba Group

We took part in the city brain challenge competition and achieved the 8th place. In this competition, the players are provided with a real-world city-scale road network and its traffic demand derived from real traffic data. The players are asked to coordinate the traffic signals with a self-designed agent to maximize the number of vehicles served while maintaining an acceptable delay. In this abstract paper, we present an overall analysis and our detailed solution to this competition. Our approach is mainly based on the adaptation of the deep Q-network (DQN) for real-time traffic signal control. From our perspective, the major challenge of this competition is how to extend the classical DQN framework to traffic signals control in real-world complex road network and traffic flow situation. After trying and implementing several classical reward functions, we finally chose to apply our newly-designed reward in our agent. By applying our newly-proposed reward function and carefully tuning the control scheme, an agent based on a single DQN model can rank among the top 15 teams. We hope this paper could serve, to some extent, as a baseline solution to traffic signal control of real-world road network and inspire further attempts and researches.


page 1

page 2

page 3

page 4


Learning Phase Competition for Traffic Signal Control

Increasingly available city data and advanced learning techniques have e...

CityFlow: A Multi-Agent Reinforcement Learning Environment for Large Scale City Traffic Scenario

Traffic signal control is an emerging application scenario for reinforce...

Traffic Light Control with Reinforcement Learning

Traffic light control is important for reducing congestion in urban mobi...

A self-organizing system for urban traffic control based on predictive interval microscopic model

This paper introduces a self-organizing traffic signal system for an urb...

Antifragile Control Systems: The case of an oscillator-based network model of urban road traffic dynamics

Existing traffic control systems only possess a local perspective over t...

Attention Gate in Traffic Forecasting

Because of increased urban complexity and growing populations, more and ...

Code Repositories

Please sign up or login with your details

Forgot password? Click here to reset