Optimizing Mixed Autonomy Traffic Flow With Decentralized Autonomous Vehicles and Multi-Agent RL

10/30/2020
by   Eugene Vinitsky, et al.
0

We study the ability of autonomous vehicles to improve the throughput of a bottleneck using a fully decentralized control scheme in a mixed autonomy setting. We consider the problem of improving the throughput of a scaled model of the San Francisco-Oakland Bay Bridge: a two-stage bottleneck where four lanes reduce to two and then reduce to one. Although there is extensive work examining variants of bottleneck control in a centralized setting, there is less study of the challenging multi-agent setting where the large number of interacting AVs leads to significant optimization difficulties for reinforcement learning methods. We apply multi-agent reinforcement algorithms to this problem and demonstrate that significant improvements in bottleneck throughput, from 20% at a 5% penetration rate to 33% at a 40% penetration rate, can be achieved. We compare our results to a hand-designed feedback controller and demonstrate that our results sharply outperform the feedback controller despite extensive tuning. Additionally, we demonstrate that the RL-based controllers adopt a robust strategy that works across penetration rates whereas the feedback controllers degrade immediately upon penetration rate variation. We investigate the feasibility of both action and observation decentralization and demonstrate that effective strategies are possible using purely local sensing. Finally, we open-source our code at https://github.com/eugenevinitsky/decentralized_bottlenecks.

READ FULL TEXT

page 1

page 4

page 5

research
02/06/2021

RIIT: Rethinking the Importance of Implementation Tricks in Multi-Agent Reinforcement Learning

In recent years, Multi-Agent Reinforcement Learning (MARL) has revolutio...
research
05/10/2023

Multi-Robot Coordination and Layout Design for Automated Warehousing

With the rapid progress in Multi-Agent Path Finding (MAPF), researchers ...
research
09/18/2023

Multi-Agent Deep Reinforcement Learning for Cooperative and Competitive Autonomous Vehicles using AutoDRIVE Ecosystem

This work presents a modular and parallelizable multi-agent deep reinfor...
research
04/29/2020

Reduced-Dimensional Reinforcement Learning Control using Singular Perturbation Approximations

We present a set of model-free, reduced-dimensional reinforcement learni...
research
06/18/2020

Efficient Ridesharing Dispatch Using Multi-Agent Reinforcement Learning

With the advent of ride-sharing services, there is a huge increase in th...
research
01/31/2019

Efficient Ridesharing Order Dispatching with Mean Field Multi-Agent Reinforcement Learning

A fundamental question in any peer-to-peer ridesharing system is how to,...

Please sign up or login with your details

Forgot password? Click here to reset