DeepAI AI Chat
Log In Sign Up

Safe Exploration by Solving Early Terminated MDP

by   Hao Sun, et al.

Safe exploration is crucial for the real-world application of reinforcement learning (RL). Previous works consider the safe exploration problem as Constrained Markov Decision Process (CMDP), where the policies are being optimized under constraints. However, when encountering any potential dangers, human tends to stop immediately and rarely learns to behave safely in danger. Motivated by human learning, we introduce a new approach to address safe RL problems under the framework of Early Terminated MDP (ET-MDP). We first define the ET-MDP as an unconstrained MDP with the same optimal value function as its corresponding CMDP. An off-policy algorithm based on context models is then proposed to solve the ET-MDP, which thereby solves the corresponding CMDP with better asymptotic performance and improved learning efficiency. Experiments on various CMDP tasks show a substantial improvement over previous methods that directly solve CMDP.


Safe Reinforcement Learning Using Advantage-Based Intervention

Many sequential decision problems involve finding a policy that maximize...

Reconnaissance and Planning algorithm for constrained MDP

Practical reinforcement learning problems are often formulated as constr...

Data Efficient and Safe Learning for Locomotion via Simplified Model

In this letter, we formulate a novel Markov Decision Process (MDP) for d...

When Blockchain Meets AI: Optimal Mining Strategy Achieved By Machine Learning

This work applies reinforcement learning (RL) from the AI machine learni...

Exploiting Multiple Abstractions in Episodic RL via Reward Shaping

One major limitation to the applicability of Reinforcement Learning (RL)...

A Reinforcement Learning Approach to the View Planning Problem

We present a Reinforcement Learning (RL) solution to the view planning p...

Enhanced Scene Specificity with Sparse Dynamic Value Estimation

Multi-scene reinforcement learning involves training the RL agent across...