DeepAI AI Chat
Log In Sign Up

Model-based Constrained Reinforcement Learning using Generalized Control Barrier Function

by   Haitong Ma, et al.

Model information can be used to predict future trajectories, so it has huge potential to avoid dangerous region when implementing reinforcement learning (RL) on real-world tasks, like autonomous driving. However, existing studies mostly use model-free constrained RL, which causes inevitable constraint violations. This paper proposes a model-based feasibility enhancement technique of constrained RL, which enhances the feasibility of policy using generalized control barrier function (GCBF) defined on the distance to constraint boundary. By using the model information, the policy can be optimized safely without violating actual safety constraints, and the sample efficiency is increased. The major difficulty of infeasibility in solving the constrained policy gradient is handled by an adaptive coefficient mechanism. We evaluate the proposed method in both simulations and real vehicle experiments in a complex autonomous driving collision avoidance task. The proposed method achieves up to four times fewer constraint violations and converges 3.36 times faster than baseline constrained RL approaches.


page 3

page 4

page 6


Guided Policy Search Model-based Reinforcement Learning for Urban Autonomous Driving

In this paper, we continue our prior work on using imitation learning (I...

Uncertainty-Aware Model-Based Reinforcement Learning with Application to Autonomous Driving

To further improve the learning efficiency and performance of reinforcem...

Safe Reinforcement Learning for Autonomous Vehicles through Parallel Constrained Policy Optimization

Reinforcement learning (RL) is attracting increasing interests in autono...

Learn Zero-Constraint-Violation Policy in Model-Free Constrained Reinforcement Learning

In the trial-and-error mechanism of reinforcement learning (RL), a notor...

Multi-vehicle Conflict Resolution in Highly Constrained Spaces by Merging Optimal Control and Reinforcement Learning

We present a novel method to address the problem of multi-vehicle confli...

Separated Proportional-Integral Lagrangian for Chance Constrained Reinforcement Learning

Safety is essential for reinforcement learning (RL) applied in real-worl...

Assured RL: Reinforcement Learning with Almost Sure Constraints

We consider the problem of finding optimal policies for a Markov Decisio...