Decision-Making under On-Ramp merge Scenarios by Distributional Soft Actor-Critic Algorithm

by   Yiting Kong, et al.

Merging into the highway from the on-ramp is an essential scenario for automated driving. The decision-making under the scenario needs to balance the safety and efficiency performance to optimize a long-term objective, which is challenging due to the dynamic, stochastic, and adversarial characteristics. The Rule-based methods often lead to conservative driving on this task while the learning-based methods have difficulties meeting the safety requirements. In this paper, we propose an RL-based end-to-end decision-making method under a framework of offline training and online correction, called the Shielded Distributional Soft Actor-critic (SDSAC). The SDSAC adopts the policy evaluation with safety consideration and a safety shield parameterized with the barrier function in its offline training and online correction, respectively. These two measures support each other for better safety while not damaging the efficiency performance severely. We verify the SDSAC on an on-ramp merge scenario in simulation. The results show that the SDSAC has the best safety performance compared to baseline algorithms and achieves efficient driving simultaneously.



page 1

page 4


Freeway Merging in Congested Traffic based on Multipolicy Decision Making with Passive Actor Critic

Freeway merging in congested traffic is a significant challenge toward f...

Encoding Distributional Soft Actor-Critic for Autonomous Driving in Multi-lane Scenarios

In this paper, we propose a new reinforcement learning (RL) algorithm, c...

SAAC: Safe Reinforcement Learning as an Adversarial Game of Actor-Critics

Although Reinforcement Learning (RL) is effective for sequential decisio...

Watch out for the risky actors: Assessing risk in dynamic environments for safe driving

Driving in a dynamic environment that consists of other actors is inhere...

A Safety-Critical Decision Making and Control Framework Combining Machine Learning and Rule-based Algorithms

While artificial-intelligence-based methods suffer from lack of transpar...

Prior Is All You Need to Improve the Robustness and Safety for the First Time Deployment of Meta RL

The field of Meta Reinforcement Learning (Meta-RL) has seen substantial ...

Soft-Robust Algorithms for Handling Model Misspecification

In reinforcement learning, robust policies for high-stakes decision-maki...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.