Autonomous Highway Merging in Mixed Traffic Using Reinforcement Learning and Motion Predictive Safety Controller

04/03/2022
by   Qianqian Liu, et al.
0

Deep reinforcement learning (DRL) has a great potential for solving complex decision-making problems in autonomous driving, especially in mixed-traffic scenarios where autonomous vehicles and human-driven vehicles (HDVs) drive together. Safety is a key during both the learning and deploying reinforcement learning (RL) algorithms process. In this paper, we formulate the on-ramp merging as a Markov Decision Process (MDP) problem and solve it with an off-policy RL algorithm, i.e., Soft Actor-Critic for Discrete Action Settings (SAC-Discrete). In addition, a motion predictive safety controller including a motion predictor and an action substitution module, is proposed to ensure driving safety during both training and testing. The motion predictor estimates the trajectories of the ego vehicle and surrounding vehicles from kinematic models, and predicts potential collisions. The action substitution module updates the actions based on safety distance and replaces risky actions, before sending them to the low-level controller. We train, evaluate and test our approach on a gym-like highway simulation with three different levels of traffic modes. The simulation results show that even in harder traffic densities, our proposed method still significantly reduces collision rate while maintaining high efficiency, outperforming several state-of-the-art baselines in the considered on-ramp merging scenarios. The video demo of the evaluation process can be found at: https://www.youtube.com/watch?v=7FvjbAM4oFw

READ FULL TEXT
research
11/11/2021

Multi-agent Reinforcement Learning for Cooperative Lane Changing of Connected and Autonomous Vehicles in Mixed Traffic

Autonomous driving has attracted significant research interests in the p...
research
08/01/2019

Learning When to Drive in Intersections by Combining Reinforcement Learning and Model Predictive Control

In this paper, we propose a decision making algorithm intended for autom...
research
07/21/2022

Towards Robust On-Ramp Merging via Augmented Multimodal Reinforcement Learning

Despite the success of AI-enabled onboard perception, on-ramp merging ha...
research
12/07/2022

RLPG: Reinforcement Learning Approach for Dynamic Intra-Platoon Gap Adaptation for Highway On-Ramp Merging

A platoon refers to a group of vehicles traveling together in very close...
research
03/30/2023

Switching Pushing Skill Combined MPC and Deep Reinforcement Learning for Planar Non-prehensile Manipulation

In this paper, a novel switching pushing skill algorithm is proposed to ...
research
08/30/2021

Integrated Decision and Control at Multi-Lane Intersections with Mixed Traffic Flow

Autonomous driving at intersections is one of the most complicated and a...

Please sign up or login with your details

Forgot password? Click here to reset