L*-Based Learning of Markov Decision Processes (Extended Version)

06/28/2019
by   Martin Tappler, et al.
0

Automata learning techniques automatically generate system models from test observations. These techniques usually fall into two categories: passive and active. Passive learning uses a predetermined data set, e.g., system logs. In contrast, active learning actively queries the system under learning, which is considered more efficient. An influential active learning technique is Angluin's L* algorithm for regular languages which inspired several generalisations from DFAs to other automata-based modelling formalisms. In this work, we study L*-based learning of deterministic Markov decision processes, first assuming an ideal setting with perfect information. Then, we relax this assumption and present a novel learning algorithm that collects information by sampling system traces via testing. Experiments with the implementation of our sampling-based algorithm suggest that it achieves better accuracy than state-of-the-art passive learning techniques with the same amount of test data. Unlike existing learning algorithms with predefined states, our algorithm learns the complete model structure including the states.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/06/2021

Active Learning of Markov Decision Processes using Baum-Welch algorithm (Extended)

Cyber-physical systems (CPSs) are naturally modelled as reactive systems...
research
10/06/2021

Active Learning for Sound Negotiations

We present two active learning algorithms for sound deterministic negoti...
research
04/01/2019

Efficient and Safe Exploration in Deterministic Markov Decision Processes with Unknown Transition Models

We propose a safe exploration algorithm for deterministic Markov Decisio...
research
01/17/2020

K-NN active learning under local smoothness assumption

There is a large body of work on convergence rates either in passive or ...
research
02/08/2019

K-nn active learning under local smoothness condition

There is a large body of work on convergence rates either in passive or ...
research
06/25/2022

Mitigating sampling bias in risk-based active learning via an EM algorithm

Risk-based active learning is an approach to developing statistical clas...
research
07/31/2022

Active Learning of One-Clock Timed Automata using Constraint Solving

Active automata learning in the framework of Angluin's L^* algorithm has...

Please sign up or login with your details

Forgot password? Click here to reset