Logically-Constrained Neural Fitted Q-Iteration

09/20/2018
by   Mohammadhosein Hasanbeig, et al.
6

This paper proposes a method for efficient training of the Q-function for continuous-state Markov Decision Processes (MDP), such that the traces of the resulting policies satisfy a Linear Temporal Logic (LTL) property. The logical property is converted into a limit deterministic Buchi automaton with which a product MDP is constructed. The control policy is then synthesized by a reinforcement learning algorithm assuming that no prior knowledge is available from the MDP. The proposed method is evaluated in a numerical study to test the quality of the generated control policy and is compared against conventional methods for policy synthesis such as MDP abstraction (Voronoi quantizer) and approximate dynamic programming (fitted value iteration).

READ FULL TEXT

page 7

page 9

page 10

page 11

research
02/02/2019

Certified Reinforcement Learning with Logic Guidance

This paper proposes the first model-free Reinforcement Learning (RL) fra...
research
04/04/2021

Reinforcement Learning with Temporal Logic Constraints for Partially-Observable Markov Decision Processes

This paper proposes a reinforcement learning method for controller synth...
research
03/29/2016

Algorithms for Batch Hierarchical Reinforcement Learning

Hierarchical Reinforcement Learning (HRL) exploits temporal abstraction ...
research
04/03/2023

Investigation of risk-aware MDP and POMDP contingency management autonomy for UAS

Unmanned aircraft systems (UAS) are being increasingly adopted for vario...
research
02/17/2021

Self-Triggered Markov Decision Processes

In this paper, we study Markov Decision Processes (MDPs) with self-trigg...
research
01/16/2015

Value Iteration with Options and State Aggregation

This paper presents a way of solving Markov Decision Processes that comb...
research
05/09/2022

Accelerated Reinforcement Learning for Temporal Logic Control Objectives

This paper addresses the problem of learning control policies for mobile...

Please sign up or login with your details

Forgot password? Click here to reset